[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:42 dist-test-slave-8tdl krb5kdc[1543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:42 dist-test-slave-8tdl krb5kdc[1543](info): set up 2 sockets
Aug 07 15:01:42 dist-test-slave-8tdl krb5kdc[1543](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:01:43.708158 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:43 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:43.855738 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:42119
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:43.971915 1557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:43.972203 1557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:43.972246 1557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:43.973686 1557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:01:43.973770 1557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:43.973798 1557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:43.973831 1557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:01:43.973855 1557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:01:43.976001 1557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:42119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:43.976512 1557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:43.976845 1557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:43.985826 1566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:43.985872 1563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:43.986205 1562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:43.986953 1557 server_base.cc:1034] running on GCE node
I20260807 15:01:43.987293 1557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:43.987720 1557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:43.988929 1557 hybrid_clock.cc:648] HybridClock initialized: now 1786114903988923 us; error 55 us; skew 500 ppm
Aug 07 15:01:43 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:43.992709 1557 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:01:43.993497 1557 webserver.cc:460] Webserver started at http://127.1.127.254:38359/ using document root <none> and password file <none>
I20260807 15:01:43.993846 1557 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:43.993930 1557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:43.994135 1557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:43.995820 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20"
format_stamp: "Formatted at 2026-08-07 15:01:43 on dist-test-slave-8tdl"
server_key: "70f0e660ddb155b2ec4593e92a786763"
server_key_iv: "f3aaa7e39b3cc59e57fdbbbdd7663369"
server_key_version: "encryptionkey@0"
I20260807 15:01:43.996492 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20"
format_stamp: "Formatted at 2026-08-07 15:01:43 on dist-test-slave-8tdl"
server_key: "70f0e660ddb155b2ec4593e92a786763"
server_key_iv: "f3aaa7e39b3cc59e57fdbbbdd7663369"
server_key_version: "encryptionkey@0"
I20260807 15:01:43.998525 1557 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:01:43.999830 1574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.000242 1557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:44.000380 1557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20"
format_stamp: "Formatted at 2026-08-07 15:01:43 on dist-test-slave-8tdl"
server_key: "70f0e660ddb155b2ec4593e92a786763"
server_key_iv: "f3aaa7e39b3cc59e57fdbbbdd7663369"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.000495 1557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:44.012418 1557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:44.015393 1557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:44.015625 1557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:44.021529 1557 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:42119
I20260807 15:01:44.021593 1627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:42119 every 8 connection(s)
I20260807 15:01:44.022240 1557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:01:44.023785 1628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:44.027318 1628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20: Bootstrap starting.
I20260807 15:01:44.028285 1628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:44.028700 1628 log.cc:824] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:44.029723 1628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20: No bootstrap required, opened a new log
I20260807 15:01:44.030921 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 1557
I20260807 15:01:44.031067 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:01:44.031876 1535 external_mini_cluster.cc:1428] Setting key 5adacc4af79b7f98c66fb9c300524d49
I20260807 15:01:44.032759 1628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } }
I20260807 15:01:44.033144 1628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:44.033272 1628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3deaa406fb8a4e1b8b2d90027ec1fc20, State: Initialized, Role: FOLLOWER
I20260807 15:01:44.033509 1628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } }
I20260807 15:01:44.033705 1628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:01:44.033762 1628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:01:44.033815 1628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:44.035161 1628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } }
I20260807 15:01:44.035388 1628 leader_election.cc:302] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3deaa406fb8a4e1b8b2d90027ec1fc20; no voters:
I20260807 15:01:44.035602 1628 leader_election.cc:288] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:01:44.035660 1633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:44.035852 1633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [term 1 LEADER]: Becoming Leader. State: Replica: 3deaa406fb8a4e1b8b2d90027ec1fc20, State: Running, Role: LEADER
I20260807 15:01:44.036008 1633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } }
I20260807 15:01:44.036201 1628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:01:44.036607 1634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } } }
I20260807 15:01:44.036579 1633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3deaa406fb8a4e1b8b2d90027ec1fc20. Latest consensus state: current_term: 1 leader_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3deaa406fb8a4e1b8b2d90027ec1fc20" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 42119 } } }
I20260807 15:01:44.036684 1634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:44.036687 1633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:44.037005 1640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:01:44.038585 1640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:01:44.041468 1640 catalog_manager.cc:1347] Generated new cluster ID: 16ce168d13c0477e92647cd2f1e8013f
I20260807 15:01:44.041554 1640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:44.052734 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.033556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.1.127.254:42119)
0807 15:01:44.033985 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.033996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.034297 (+ 301us) server_negotiation.cc:407] Connection header received
0807 15:01:44.036601 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.036609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.036718 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.036902 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.039673 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.041117 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.042856 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.043519 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.043595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.048193 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.048199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.048200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.048214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.050326 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.050923 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.050927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.050927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.051048 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.051319 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.051320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.051320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.051636 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.051783 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.052196 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.052418 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":133,"threads_started":1}
I20260807 15:01:44.061481 1640 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:01:44.062098 1640 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:01:44.067057 1640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3deaa406fb8a4e1b8b2d90027ec1fc20: Generated new TSK 0
I20260807 15:01:44.067278 1640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:01:44.112464 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:44.253312 1657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:44.253623 1657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:44.253675 1657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:44.255374 1657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:44.255465 1657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:44.255525 1657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:01:44.257519 1657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:44.257930 1657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:44.258219 1657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:44.266793 1662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.266848 1663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.267004 1665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:44.267494 1657 server_base.cc:1034] running on GCE node
I20260807 15:01:44.267726 1657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:44.268046 1657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:44.269232 1657 hybrid_clock.cc:648] HybridClock initialized: now 1786114904269213 us; error 34 us; skew 500 ppm
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:44.272675 1657 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:01:44.273326 1657 webserver.cc:460] Webserver started at http://127.1.127.193:36033/ using document root <none> and password file <none>
I20260807 15:01:44.273602 1657 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:44.273676 1657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:44.273828 1657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:44.275123 1657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "11eb8536dd2f43738e4fc4be82ddf958"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "6c5b69c1f89629467c39bb198b946317"
server_key_iv: "4ede079df94ae7102f37e39f7940df55"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.275563 1657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "11eb8536dd2f43738e4fc4be82ddf958"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "6c5b69c1f89629467c39bb198b946317"
server_key_iv: "4ede079df94ae7102f37e39f7940df55"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.277604 1657 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:01:44.278867 1674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.279153 1657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:44.279273 1657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "11eb8536dd2f43738e4fc4be82ddf958"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "6c5b69c1f89629467c39bb198b946317"
server_key_iv: "4ede079df94ae7102f37e39f7940df55"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.279333 1657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:44.292654 1657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:44.295756 1657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:44.295962 1657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:44.296396 1657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:44.296985 1657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:44.297101 1657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.297138 1657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:44.297158 1657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.308504 1657 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:46877
I20260807 15:01:44.308683 1793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:46877 every 8 connection(s)
I20260807 15:01:44.309005 1657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:01:44.312218 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 1657
I20260807 15:01:44.312299 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:01:44.312477 1535 external_mini_cluster.cc:1428] Setting key 467143ebd2bc036c56139133a1be493d
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:44.323969 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.310835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:44835 (local address 127.1.127.254:42119)
0807 15:01:44.310920 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.310923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.311186 (+ 263us) server_negotiation.cc:407] Connection header received
0807 15:01:44.312111 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.312113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.312128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.312188 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.313881 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.314468 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.316073 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.316280 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.316524 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.319893 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.319896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.319896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.319907 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.322088 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.322616 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.322618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.322618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.322731 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.323100 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.323101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.323102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.323278 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.323372 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.323706 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.323835 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:01:44.324012 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.310534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:42119 (local address 127.1.127.193:44835)
0807 15:01:44.310974 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:44.311030 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:44.311968 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:44.312280 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:44.312285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:44.313384 (+ 1099us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:44.313767 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.313775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.314584 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.314585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.315940 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.315954 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.316551 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.316552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.316804 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.317564 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:44.317576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:44.319794 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:44.322211 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.322220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.322222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.322497 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.322842 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.322843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.322843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.322988 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.323363 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:44.323370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:44.323531 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:01:44.323629 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:01:44.323785 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":220,"threads_started":1}
I20260807 15:01:44.324862 1794 heartbeater.cc:347] Connected to a master server at 127.1.127.254:42119
I20260807 15:01:44.325017 1794 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:44.325281 1794 heartbeater.cc:511] Master 127.1.127.254:42119 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:01:44.326611 1592 ts_manager.cc:194] Registered new tserver with Master: 11eb8536dd2f43738e4fc4be82ddf958 (127.1.127.193:46877)
I20260807 15:01:44.327549 1592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:44835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:01:44.351783 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:44.481426 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:44.481673 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:44.481702 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:44.483482 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:44.483543 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:44.483577 1799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:01:44.485518 1799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:44.485747 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:44.485958 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:44.492406 1805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.492590 1804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.492441 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:44.492758 1799 server_base.cc:1034] running on GCE node
I20260807 15:01:44.493031 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:44.493325 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:44.494529 1799 hybrid_clock.cc:648] HybridClock initialized: now 1786114904494482 us; error 62 us; skew 500 ppm
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:44.497485 1799 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:01:44.497991 1799 webserver.cc:460] Webserver started at http://127.1.127.194:44923/ using document root <none> and password file <none>
I20260807 15:01:44.498216 1799 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:44.498260 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:44.498356 1799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:44.499624 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "c27a3801fc8d46a58ece07e2a1af332b"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "9aba97f523aba694b2ef5c28a9efa349"
server_key_iv: "0780c4a5bdff07f5115369e9e9637ba3"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.500010 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "c27a3801fc8d46a58ece07e2a1af332b"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "9aba97f523aba694b2ef5c28a9efa349"
server_key_iv: "0780c4a5bdff07f5115369e9e9637ba3"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.501408 1799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:01:44.502326 1814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.502704 1799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:44.502795 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "c27a3801fc8d46a58ece07e2a1af332b"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "9aba97f523aba694b2ef5c28a9efa349"
server_key_iv: "0780c4a5bdff07f5115369e9e9637ba3"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.502857 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:44.512748 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:44.515595 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:44.515722 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:44.515972 1799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:44.516347 1799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:44.516386 1799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.516410 1799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:44.516448 1799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.531728 1799 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:34895
I20260807 15:01:44.531769 1933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:34895 every 8 connection(s)
I20260807 15:01:44.532228 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:01:44.539654 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 1799
I20260807 15:01:44.539757 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:01:44.540062 1535 external_mini_cluster.cc:1428] Setting key b090bddf09818cbe98c5760283c58963
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:44.546295 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.533262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:41631 (local address 127.1.127.254:42119)
0807 15:01:44.533347 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.533351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.533794 (+ 443us) server_negotiation.cc:407] Connection header received
0807 15:01:44.534865 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.534867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.534880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.534945 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.536825 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.537456 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.538839 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.539169 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.539234 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.542455 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.542457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.542458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.542478 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.544471 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.545140 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.545141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.545141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.545206 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.545541 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.545542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.545542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.545665 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.545725 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.546134 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.546221 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:01:44.546849 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.533233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:42119 (local address 127.1.127.194:41631)
0807 15:01:44.533631 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:44.533675 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:44.534638 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:44.535032 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:44.535036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:44.536246 (+ 1210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:44.536733 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.536740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.537596 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.537597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.538750 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.538754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.539275 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.539276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.539542 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.540410 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:44.540420 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:44.542348 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:44.544624 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.544634 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.544637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.545041 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.545289 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.545290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.545291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.545466 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.545740 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:44.545748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:44.545952 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 15:01:44.546111 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 15:01:44.546362 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":216,"threads_started":1}
I20260807 15:01:44.547696 1934 heartbeater.cc:347] Connected to a master server at 127.1.127.254:42119
I20260807 15:01:44.547829 1934 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:44.548074 1934 heartbeater.cc:511] Master 127.1.127.254:42119 requested a full tablet report, sending...
I20260807 15:01:44.548578 1592 ts_manager.cc:194] Registered new tserver with Master: c27a3801fc8d46a58ece07e2a1af332b (127.1.127.194:34895)
I20260807 15:01:44.549032 1592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:41631
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:01:44.575282 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:44.696192 1940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:44.696449 1940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:44.696480 1940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:44.697932 1940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:44.697990 1940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:44.698031 1940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:01:44.700011 1940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:42119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:44.700309 1940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:44.700560 1940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:44.707228 1946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.707358 1947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:44.707276 1949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:44.707551 1940 server_base.cc:1034] running on GCE node
I20260807 15:01:44.707788 1940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:44.708139 1940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:44.709304 1940 hybrid_clock.cc:648] HybridClock initialized: now 1786114904709286 us; error 41 us; skew 500 ppm
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:44.712945 1940 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:01:44.713503 1940 webserver.cc:460] Webserver started at http://127.1.127.195:43997/ using document root <none> and password file <none>
I20260807 15:01:44.713770 1940 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:44.713826 1940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:44.713971 1940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:44.715507 1940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "4ede752fb5a54d9e9e340b579bf41bc1"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "3186087c64200e359c73be0e0e9b5edd"
server_key_iv: "9d1a4f4ae1fdf6c6219c096cd9b8926a"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.715950 1940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "4ede752fb5a54d9e9e340b579bf41bc1"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "3186087c64200e359c73be0e0e9b5edd"
server_key_iv: "9d1a4f4ae1fdf6c6219c096cd9b8926a"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.717633 1940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:44.718899 1956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.719182 1940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:44.719295 1940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "4ede752fb5a54d9e9e340b579bf41bc1"
format_stamp: "Formatted at 2026-08-07 15:01:44 on dist-test-slave-8tdl"
server_key: "3186087c64200e359c73be0e0e9b5edd"
server_key_iv: "9d1a4f4ae1fdf6c6219c096cd9b8926a"
server_key_version: "encryptionkey@0"
I20260807 15:01:44.719388 1940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:44.728804 1940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:44.731897 1940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:44.732131 1940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:44.732574 1940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:44.733027 1940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:44.733088 1940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.733122 1940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:44.733153 1940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:44.751538 1940 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:43567
I20260807 15:01:44.751732 2074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:43567 every 8 connection(s)
I20260807 15:01:44.752099 1940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:01:44.753448 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 1940
I20260807 15:01:44.753634 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:01:44.753849 1535 external_mini_cluster.cc:1428] Setting key 1bac22564e0a241fb659942424b174f7
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:44.764914 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.753265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:38277 (local address 127.1.127.254:42119)
0807 15:01:44.753355 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.753358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.753816 (+ 458us) server_negotiation.cc:407] Connection header received
0807 15:01:44.754904 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.754905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.754920 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.754986 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.756485 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.757040 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.758287 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.758487 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.758574 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.761698 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.761700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.761701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.761709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.763349 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.763820 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.763821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.763821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.763880 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.764186 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.764186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.764187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.764305 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.764361 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.764751 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.764839 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:01:44.765213 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.753173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:42119 (local address 127.1.127.195:38277)
0807 15:01:44.753631 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:44.753687 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:44.754756 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:44.755149 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:44.755153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:44.756050 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:44.756372 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.756378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.757146 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.757147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.758178 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:44.758183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.758620 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.758621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:44.758901 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.759671 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:44.759680 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:44.761578 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:44.763460 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.763470 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.763472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.763738 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.763943 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:44.763944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:44.763944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:44.764120 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:44.764364 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:44.764369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:44.764506 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:01:44.764598 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:01:44.764748 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260807 15:01:44.765929 2075 heartbeater.cc:347] Connected to a master server at 127.1.127.254:42119
I20260807 15:01:44.766031 2075 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:44.766247 2075 heartbeater.cc:511] Master 127.1.127.254:42119 requested a full tablet report, sending...
I20260807 15:01:44.766788 1590 ts_manager.cc:194] Registered new tserver with Master: 4ede752fb5a54d9e9e340b579bf41bc1 (127.1.127.195:43567)
I20260807 15:01:44.767200 1590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:38277
I20260807 15:01:44.778167 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:44.808290 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.797984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.1.127.193:46877)
0807 15:01:44.798142 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.798165 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.798293 (+ 128us) server_negotiation.cc:407] Connection header received
0807 15:01:44.798412 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.798416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.798437 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.798563 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.799348 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.800134 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.800955 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.801420 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.801512 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.804206 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.804216 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.804216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.804224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.806788 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.807218 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.807222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.807223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.807292 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.807566 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.807568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.807568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.807738 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.808043 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.808087 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.808166 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
W20260807 15:01:44.808903 1703 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56476
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[1543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:44.836910 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:44.828405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.1.127.193:46877)
0807 15:01:44.828503 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:44.828506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:44.828599 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:01:44.828704 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:44.828705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:44.828722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:44.828798 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:44.829485 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.830037 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.830869 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:44.831094 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:44.831176 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:44.833706 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:44.833708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:44.833709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:44.833724 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:44.835517 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.836039 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.836046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.836046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.836105 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:44.836323 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:44.836324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:44.836325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:44.836453 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:44.836568 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:44.836680 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:01:44.836779 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:01:44.837617 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 1657
I20260807 15:01:44.844352 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 1799
I20260807 15:01:44.851495 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 1940
I20260807 15:01:44.857147 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 1557
2026-08-07T15:01:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2293 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[2090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[2090](info): set up 2 sockets
Aug 07 15:01:44 dist-test-slave-8tdl krb5kdc[2090](info): commencing operation
krb5kdc: starting...
W20260807 15:01:45.975308 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:46.115113 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37939
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:45577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:46.202217 2105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:46.202486 2105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:46.202528 2105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:46.203801 2105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:01:46.203883 2105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:46.203908 2105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:46.203930 2105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:01:46.203960 2105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:01:46.205744 2105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:46.206077 2105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:46.206336 2105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:46.211525 2111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.211563 2113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.211531 2110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:46.212729 2105 server_base.cc:1034] running on GCE node
I20260807 15:01:46.212997 2105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:46.213311 2105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:46.214490 2105 hybrid_clock.cc:648] HybridClock initialized: now 1786114906214483 us; error 39 us; skew 500 ppm
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:46.217666 2105 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:01:46.218279 2105 webserver.cc:460] Webserver started at http://127.1.127.254:43165/ using document root <none> and password file <none>
I20260807 15:01:46.218619 2105 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:46.218719 2105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:46.218866 2105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:46.220072 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "053d03019e8f4c17b4ec54717409e6b8"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "afc929ced8be6add09dcae59e310a9ca"
server_key_iv: "04d2676447bb0f184abe7839036df720"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.220474 2105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "053d03019e8f4c17b4ec54717409e6b8"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "afc929ced8be6add09dcae59e310a9ca"
server_key_iv: "04d2676447bb0f184abe7839036df720"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.222187 2105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:01:46.223349 2120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.223590 2105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:01:46.223690 2105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "053d03019e8f4c17b4ec54717409e6b8"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "afc929ced8be6add09dcae59e310a9ca"
server_key_iv: "04d2676447bb0f184abe7839036df720"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.223752 2105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:46.240408 2105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:46.243010 2105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:46.243144 2105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:46.247435 2105 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:37939
I20260807 15:01:46.247865 2105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:01:46.247906 2173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:37939 every 8 connection(s)
I20260807 15:01:46.248628 2174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:46.251343 2174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8: Bootstrap starting.
I20260807 15:01:46.251679 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2105
I20260807 15:01:46.251766 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:01:46.252005 1535 external_mini_cluster.cc:1428] Setting key 85e303e4f29440f723f68473c93a83e0
I20260807 15:01:46.252182 2174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:46.252609 2174 log.cc:824] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:46.253508 2174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8: No bootstrap required, opened a new log
I20260807 15:01:46.256691 2174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } }
I20260807 15:01:46.256929 2174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:46.256973 2174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053d03019e8f4c17b4ec54717409e6b8, State: Initialized, Role: FOLLOWER
I20260807 15:01:46.257141 2174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } }
I20260807 15:01:46.257246 2174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:01:46.257301 2174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:01:46.257377 2174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:46.258260 2174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } }
I20260807 15:01:46.258399 2174 leader_election.cc:302] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053d03019e8f4c17b4ec54717409e6b8; no voters:
I20260807 15:01:46.258630 2174 leader_election.cc:288] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:01:46.258741 2180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:46.258987 2180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [term 1 LEADER]: Becoming Leader. State: Replica: 053d03019e8f4c17b4ec54717409e6b8, State: Running, Role: LEADER
I20260807 15:01:46.259123 2180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } }
I20260807 15:01:46.258980 2174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:01:46.259574 2181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "053d03019e8f4c17b4ec54717409e6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } } }
I20260807 15:01:46.259877 2181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:46.259789 2182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053d03019e8f4c17b4ec54717409e6b8. Latest consensus state: current_term: 1 leader_uuid: "053d03019e8f4c17b4ec54717409e6b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053d03019e8f4c17b4ec54717409e6b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37939 } } }
I20260807 15:01:46.260170 2182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:46.260421 2184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:46.261092 2184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:01:46.263553 2184 catalog_manager.cc:1347] Generated new cluster ID: 98b28468fca840eb947082b1c304518b
I20260807 15:01:46.263612 2184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:01:46.265703 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.252690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.1.127.254:37939)
0807 15:01:46.252941 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.252953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.252994 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:01:46.253565 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.253574 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.253675 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.253840 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.254798 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.255811 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.257172 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.257779 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.257878 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.261540 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.261545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.261546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.261561 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.263927 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.264340 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.264343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.264344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.264468 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.264667 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.264667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.264668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.264958 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.265084 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.265332 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.265483 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":110,"threads_started":1}
I20260807 15:01:46.272209 2184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:01:46.272773 2184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:01:46.278720 2184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053d03019e8f4c17b4ec54717409e6b8: Generated new TSK 0
I20260807 15:01:46.278914 2184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:01:46.316216 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:45577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:46.406534 2202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:46.406813 2202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:46.406843 2202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:46.408169 2202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:46.408217 2202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:46.408248 2202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:01:46.410019 2202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:46.410301 2202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:46.410589 2202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:46.416246 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.416276 2211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.416246 2207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:46.416535 2202 server_base.cc:1034] running on GCE node
I20260807 15:01:46.416800 2202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:46.417131 2202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:46.418299 2202 hybrid_clock.cc:648] HybridClock initialized: now 1786114906418283 us; error 32 us; skew 500 ppm
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:46.421386 2202 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:01:46.421926 2202 webserver.cc:460] Webserver started at http://127.1.127.193:34357/ using document root <none> and password file <none>
I20260807 15:01:46.422223 2202 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:46.422268 2202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:46.422366 2202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:46.423525 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "86b0448dd4b744b5a271e6fffb85e20d"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "b2ff091a41c1bce08551d1221edbe02b"
server_key_iv: "48a1272d7d061d7dee04babb097d9c67"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.423851 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "86b0448dd4b744b5a271e6fffb85e20d"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "b2ff091a41c1bce08551d1221edbe02b"
server_key_iv: "48a1272d7d061d7dee04babb097d9c67"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.425299 2202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:01:46.426225 2218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.426461 2202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:46.426759 2202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "86b0448dd4b744b5a271e6fffb85e20d"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "b2ff091a41c1bce08551d1221edbe02b"
server_key_iv: "48a1272d7d061d7dee04babb097d9c67"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.426852 2202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:46.449301 2202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:46.451882 2202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:46.452059 2202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:46.452468 2202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:46.452840 2202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:46.452919 2202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.452958 2202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:46.453006 2202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.460283 2202 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:39129
I20260807 15:01:46.460352 2336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:39129 every 8 connection(s)
I20260807 15:01:46.460709 2202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:01:46.463271 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2202
I20260807 15:01:46.463394 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:01:46.463577 1535 external_mini_cluster.cc:1428] Setting key 98d523306beb96caaf7bfb0834f1ca01
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:01:46.472735 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.461577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34181 (local address 127.1.127.254:37939)
0807 15:01:46.461646 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.461648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.461884 (+ 236us) server_negotiation.cc:407] Connection header received
0807 15:01:46.463127 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.463130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.463146 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.463231 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.464257 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.464988 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.466347 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.466631 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.466739 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.469382 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.469383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.469384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.469393 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.471163 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.471638 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.471639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.471639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.471706 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.471998 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.471999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.471999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.472146 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.472231 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.472531 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.472636 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:01:46.472957 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.461570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37939 (local address 127.1.127.193:34181)
0807 15:01:46.461756 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.461794 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.462982 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.463336 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.463341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.463775 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:46.464167 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.464174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.465095 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.465096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.466206 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.466210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.466749 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.466751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.466923 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.467520 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:46.467530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:46.469258 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:46.471258 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.471268 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.471271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.471540 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.471773 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.471774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.471774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.471896 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.472220 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:46.472228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:46.472386 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.472510 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.472685 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260807 15:01:46.473788 2337 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37939
I20260807 15:01:46.473973 2337 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:46.474246 2337 heartbeater.cc:511] Master 127.1.127.254:37939 requested a full tablet report, sending...
I20260807 15:01:46.475013 2138 ts_manager.cc:194] Registered new tserver with Master: 86b0448dd4b744b5a271e6fffb85e20d (127.1.127.193:39129)
I20260807 15:01:46.476049 2138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:34181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:01:46.499066 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:45577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:46.590006 2342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:46.590255 2342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:46.590286 2342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:46.591617 2342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:46.591667 2342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:46.591698 2342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:01:46.593432 2342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:46.593660 2342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:46.593887 2342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:01:46.600253 2342 server_base.cc:1034] running on GCE node
W20260807 15:01:46.600242 2348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.600242 2347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.600412 2350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:46.600805 2342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:46.601100 2342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:46.602267 2342 hybrid_clock.cc:648] HybridClock initialized: now 1786114906602219 us; error 68 us; skew 500 ppm
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:46.605252 2342 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:01:46.605796 2342 webserver.cc:460] Webserver started at http://127.1.127.194:38961/ using document root <none> and password file <none>
I20260807 15:01:46.606084 2342 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:46.606168 2342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:46.606345 2342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:46.607687 2342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "0955c4f88cf748acbafe7c0a8c44c745"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "0af6aaf973fbd915f5283a7cdde9c43e"
server_key_iv: "9ba18ee6d6134d316a16a9a468a1d09f"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.608183 2342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "0955c4f88cf748acbafe7c0a8c44c745"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "0af6aaf973fbd915f5283a7cdde9c43e"
server_key_iv: "9ba18ee6d6134d316a16a9a468a1d09f"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.609757 2342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:01:46.610795 2358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.611091 2342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:46.611210 2342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "0955c4f88cf748acbafe7c0a8c44c745"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "0af6aaf973fbd915f5283a7cdde9c43e"
server_key_iv: "9ba18ee6d6134d316a16a9a468a1d09f"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.611308 2342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:46.622510 2342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:46.625167 2342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:46.625365 2342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:46.625664 2342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:46.626111 2342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:46.626183 2342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.626230 2342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:46.626271 2342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.633049 2342 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:46341
I20260807 15:01:46.633111 2479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:46341 every 8 connection(s)
I20260807 15:01:46.633499 2342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:01:46.635927 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2342
I20260807 15:01:46.636050 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:01:46.636231 1535 external_mini_cluster.cc:1428] Setting key 20dc80d359d1f33fdf021056f7c3ee14
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:46.646147 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.634339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:44727 (local address 127.1.127.254:37939)
0807 15:01:46.634412 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.634414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.634636 (+ 222us) server_negotiation.cc:407] Connection header received
0807 15:01:46.635743 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.635744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.635758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.635815 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.636661 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.637288 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.638459 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.638832 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.638918 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.642167 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.642169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.642170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.642182 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.644281 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.644830 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.644833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.644833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.644914 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.645272 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.645273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.645274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.645463 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.645550 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.645890 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.646008 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:01:46.646327 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.634311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37939 (local address 127.1.127.194:44727)
0807 15:01:46.634509 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.634548 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.635594 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.635895 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.635898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.636235 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:46.636558 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.636566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.637414 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.637415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.638349 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.638352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.638949 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.638950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.639128 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.639741 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:46.639753 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:46.641973 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:46.644432 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.644446 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.644448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.644734 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.645023 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.645024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.645024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.645183 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.645557 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:46.645566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:46.645722 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.645828 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.646066 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260807 15:01:46.647078 2480 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37939
I20260807 15:01:46.647226 2480 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:46.647464 2480 heartbeater.cc:511] Master 127.1.127.254:37939 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:01:46.648069 2138 ts_manager.cc:194] Registered new tserver with Master: 0955c4f88cf748acbafe7c0a8c44c745 (127.1.127.194:46341)
I20260807 15:01:46.648502 2138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:44727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:01:46.671440 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:45577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:46.759549 2486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:46.759830 2486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:46.759874 2486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:46.761303 2486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:46.761353 2486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:46.761382 2486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:01:46.763082 2486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:46.763351 2486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:46.763628 2486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:46.769277 2493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:46.769409 2486 server_base.cc:1034] running on GCE node
W20260807 15:01:46.769272 2492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:46.769277 2495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:46.769878 2486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:46.770180 2486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:46.771361 2486 hybrid_clock.cc:648] HybridClock initialized: now 1786114906771328 us; error 52 us; skew 500 ppm
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:46.774485 2486 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:01:46.775164 2486 webserver.cc:460] Webserver started at http://127.1.127.195:45169/ using document root <none> and password file <none>
I20260807 15:01:46.775449 2486 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:46.775532 2486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:46.775704 2486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:46.776961 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "a283eabbf21d487b805495f923dc8245"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "bfe730730d9be06be822446afe93af41"
server_key_iv: "87124d4493d5153d145d40b5e4c1c94e"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.777437 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "a283eabbf21d487b805495f923dc8245"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "bfe730730d9be06be822446afe93af41"
server_key_iv: "87124d4493d5153d145d40b5e4c1c94e"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.779173 2486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260807 15:01:46.780175 2502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.780458 2486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:46.780608 2486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "a283eabbf21d487b805495f923dc8245"
format_stamp: "Formatted at 2026-08-07 15:01:46 on dist-test-slave-8tdl"
server_key: "bfe730730d9be06be822446afe93af41"
server_key_iv: "87124d4493d5153d145d40b5e4c1c94e"
server_key_version: "encryptionkey@0"
I20260807 15:01:46.780755 2486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:46.794509 2486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:46.797425 2486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:46.797662 2486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:46.797966 2486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:46.798319 2486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:46.798385 2486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.798429 2486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:46.798460 2486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:46.805411 2486 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:37517
I20260807 15:01:46.805459 2621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:37517 every 8 connection(s)
I20260807 15:01:46.805863 2486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:01:46.807914 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2486
I20260807 15:01:46.808115 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:01:46.808338 1535 external_mini_cluster.cc:1428] Setting key 95cd1a5927b1ca41c2086e40d4b9856b
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:46.817688 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.806860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:58807 (local address 127.1.127.254:37939)
0807 15:01:46.806958 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.806961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.807235 (+ 274us) server_negotiation.cc:407] Connection header received
0807 15:01:46.808608 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.808610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.808624 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.808698 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.809801 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.810384 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.811638 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.811848 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.811905 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.814482 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.814483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.814483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.814491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.816107 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.816599 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.816600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.816601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.816659 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.816972 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.816974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.816974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.817105 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.817174 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.817538 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.817617 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:01:46.817879 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.806858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37939 (local address 127.1.127.195:58807)
0807 15:01:46.807094 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.807136 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.808453 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.808785 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.808790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.809404 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:46.809713 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.809719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.810498 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.810499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.811505 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.811508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.811985 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.811986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.812101 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.812745 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:46.812755 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:46.814383 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:46.816211 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.816218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.816220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.816517 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.816729 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:46.816729 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:46.816729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:46.816849 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:46.817211 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:46.817219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:46.817370 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.817477 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.817619 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":122,"threads_started":1}
I20260807 15:01:46.818606 2622 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37939
I20260807 15:01:46.818770 2622 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:46.819062 2622 heartbeater.cc:511] Master 127.1.127.254:37939 requested a full tablet report, sending...
I20260807 15:01:46.819618 2138 ts_manager.cc:194] Registered new tserver with Master: a283eabbf21d487b805495f923dc8245 (127.1.127.195:37517)
I20260807 15:01:46.820060 2138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:58807
I20260807 15:01:46.820519 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:01:46.829011 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.822596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.1.127.254:37939)
0807 15:01:46.822709 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.822712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.822852 (+ 140us) server_negotiation.cc:407] Connection header received
0807 15:01:46.822878 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.822878 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.822888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.822949 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.823502 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.823998 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.824738 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.824921 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.824987 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.826279 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.826281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.826281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.826289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.827857 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.828223 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.828224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.828224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.828284 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.828446 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.828447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.828447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.828600 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.828679 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.828753 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.828819 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:01:46.832446 2137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:46.834465 2137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:46.843526 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.838594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.1.127.194:46341)
0807 15:01:46.839185 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.839189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.839199 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:46.839243 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.839245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.839262 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.839353 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:46.839772 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.840879 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.842090 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.843020 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.843130 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.843203 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.843392 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.843435 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":145,"threads_started":1}
I20260807 15:01:46.843578 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.838461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:37517 (local address 127.0.0.1:60350)
0807 15:01:46.838828 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.838876 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.838986 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.839260 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.839264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.839266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:46.839364 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.839367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.840906 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.840907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.842108 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.842111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.843243 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.843245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.843303 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.843306 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.843377 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.843434 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":9,"thread_start_us":69,"threads_started":1}
I20260807 15:01:46.843638 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.838572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.1.127.195:37517)
0807 15:01:46.838809 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.838815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.838949 (+ 134us) server_negotiation.cc:407] Connection header received
0807 15:01:46.839098 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.839101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.839145 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.839228 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:46.839456 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.840695 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.842185 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.843084 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.843146 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.843203 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.843485 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.843531 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":133,"threads_started":1}
I20260807 15:01:46.843869 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.838471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:46341 (local address 127.0.0.1:34260)
0807 15:01:46.838849 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.838883 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.838965 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.839450 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.839451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.839453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:46.839668 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.839672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.841076 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.841077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.842000 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.842003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.843139 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.843140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.843224 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.843244 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.843356 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.843429 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":90,"threads_started":1}
I20260807 15:01:46.844816 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.838763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39129 (local address 127.0.0.1:36418)
0807 15:01:46.838901 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.838911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.838965 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.839384 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.839385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.839387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:46.839495 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.839497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.840695 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.840696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.842108 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.842111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.844511 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.844512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.844587 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.844592 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.844674 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.844728 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":59,"thread_start_us":100,"threads_started":1}
I20260807 15:01:46.844879 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.839034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.1.127.193:39129)
0807 15:01:46.839229 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.839233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.839244 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:01:46.839284 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.839286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.839301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.839371 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:46.839561 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.840490 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.842779 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.843562 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.844507 (+ 945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.844610 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.844691 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.844744 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260807 15:01:46.844780 2552 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc628ba12524bd984c40f5887a2a305 (DEFAULT_TABLE table=test-table [id=ae9581b7a4af43c083dc5dfc0046951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:46.845003 2410 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc628ba12524bd984c40f5887a2a305 (DEFAULT_TABLE table=test-table [id=ae9581b7a4af43c083dc5dfc0046951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:46.845245 2552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc628ba12524bd984c40f5887a2a305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:46.845541 2410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc628ba12524bd984c40f5887a2a305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:46.845705 2268 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc628ba12524bd984c40f5887a2a305 (DEFAULT_TABLE table=test-table [id=ae9581b7a4af43c083dc5dfc0046951d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:46.846009 2268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc628ba12524bd984c40f5887a2a305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:46.850095 2641 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Bootstrap starting.
I20260807 15:01:46.851050 2642 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Bootstrap starting.
I20260807 15:01:46.852098 2642 tablet_bootstrap.cc:654] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:46.852119 2641 tablet_bootstrap.cc:654] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:46.852996 2642 log.cc:824] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:46.853448 2643 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: Bootstrap starting.
I20260807 15:01:46.854102 2642 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: No bootstrap required, opened a new log
I20260807 15:01:46.854161 2643 tablet_bootstrap.cc:654] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:46.854210 2642 ts_tablet_manager.cc:1397] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:01:46.854910 2643 log.cc:824] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:46.854911 2641 log.cc:824] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:46.855986 2643 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: No bootstrap required, opened a new log
I20260807 15:01:46.855979 2641 tablet_bootstrap.cc:492] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: No bootstrap required, opened a new log
I20260807 15:01:46.856073 2641 ts_tablet_manager.cc:1397] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260807 15:01:46.856073 2643 ts_tablet_manager.cc:1397] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:01:46.856472 2642 raft_consensus.cc:348] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.856664 2642 raft_consensus.cc:374] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:46.856710 2642 raft_consensus.cc:729] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0955c4f88cf748acbafe7c0a8c44c745, State: Initialized, Role: FOLLOWER
I20260807 15:01:46.856837 2642 consensus_queue.cc:260] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.857231 2642 ts_tablet_manager.cc:1428] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:01:46.857266 2480 heartbeater.cc:503] Master 127.1.127.254:37939 was elected leader, sending a full tablet report...
I20260807 15:01:46.858629 2643 raft_consensus.cc:348] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.858546 2641 raft_consensus.cc:348] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.858858 2641 raft_consensus.cc:374] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:46.859160 2641 raft_consensus.cc:729] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b0448dd4b744b5a271e6fffb85e20d, State: Initialized, Role: FOLLOWER
I20260807 15:01:46.858858 2643 raft_consensus.cc:374] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:46.859243 2643 raft_consensus.cc:729] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a283eabbf21d487b805495f923dc8245, State: Initialized, Role: FOLLOWER
I20260807 15:01:46.859383 2643 consensus_queue.cc:260] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.859368 2641 consensus_queue.cc:260] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.859771 2643 ts_tablet_manager.cc:1428] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:01:46.859771 2641 ts_tablet_manager.cc:1428] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:01:46.859808 2337 heartbeater.cc:503] Master 127.1.127.254:37939 was elected leader, sending a full tablet report...
I20260807 15:01:46.860237 2622 heartbeater.cc:503] Master 127.1.127.254:37939 was elected leader, sending a full tablet report...
I20260807 15:01:46.881255 2647 raft_consensus.cc:482] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:46.881394 2647 raft_consensus.cc:504] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.882114 2647 leader_election.cc:288] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a283eabbf21d487b805495f923dc8245 (127.1.127.195:37517), 86b0448dd4b744b5a271e6fffb85e20d (127.1.127.193:39129)
W20260807 15:01:46.884192 2482 tablet.cc:2369] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:46.885716 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.882397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:37517 (local address 127.1.127.194:42489)
0807 15:01:46.882512 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.882528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.882597 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.882809 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.882810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.882812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:46.882924 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.882926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.883722 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.883723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.884854 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.884857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.885448 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.885449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.885536 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.885541 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.885598 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.885651 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 15:01:46.885746 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.882531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:42489 (local address 127.1.127.195:37517)
0807 15:01:46.882610 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.882612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.882626 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:01:46.882660 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.882661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.882700 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.882755 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:46.883023 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.883601 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.884947 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.885363 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.885421 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.885429 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.885622 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.885663 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:01:46.885896 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.882583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39129 (local address 127.1.127.194:59527)
0807 15:01:46.882810 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:46.882819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:46.882877 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:46.883018 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:46.883019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:46.883021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:46.883094 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.883095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.883836 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.883837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.884917 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:46.884920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.885628 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.885629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:46.885699 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.885703 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:46.885761 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:01:46.885808 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":149,"threads_started":1}
I20260807 15:01:46.885938 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.882721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:59527 (local address 127.1.127.193:39129)
0807 15:01:46.882771 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.882774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.882893 (+ 119us) server_negotiation.cc:407] Connection header received
0807 15:01:46.882925 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.882926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.882940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.882997 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:46.883191 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.883753 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.885010 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.885512 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.885595 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.885605 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.885789 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.885844 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:01:46.886264 2573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc628ba12524bd984c40f5887a2a305" candidate_uuid: "0955c4f88cf748acbafe7c0a8c44c745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a283eabbf21d487b805495f923dc8245" is_pre_election: true
I20260807 15:01:46.886471 2573 raft_consensus.cc:2454] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0955c4f88cf748acbafe7c0a8c44c745 in term 0.
I20260807 15:01:46.886778 2362 leader_election.cc:302] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0955c4f88cf748acbafe7c0a8c44c745, a283eabbf21d487b805495f923dc8245; no voters:
I20260807 15:01:46.887014 2647 raft_consensus.cc:2781] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:46.887102 2647 raft_consensus.cc:482] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:46.887137 2647 raft_consensus.cc:3037] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:46.887964 2647 raft_consensus.cc:504] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.888135 2647 leader_election.cc:288] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [CANDIDATE]: Term 1 election: Requested vote from peers a283eabbf21d487b805495f923dc8245 (127.1.127.195:37517), 86b0448dd4b744b5a271e6fffb85e20d (127.1.127.193:39129)
I20260807 15:01:46.888293 2573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc628ba12524bd984c40f5887a2a305" candidate_uuid: "0955c4f88cf748acbafe7c0a8c44c745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a283eabbf21d487b805495f923dc8245"
I20260807 15:01:46.888386 2573 raft_consensus.cc:3037] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:46.888403 2288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc628ba12524bd984c40f5887a2a305" candidate_uuid: "0955c4f88cf748acbafe7c0a8c44c745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b0448dd4b744b5a271e6fffb85e20d"
I20260807 15:01:46.888588 2288 raft_consensus.cc:3037] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:46.886267 2289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc628ba12524bd984c40f5887a2a305" candidate_uuid: "0955c4f88cf748acbafe7c0a8c44c745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b0448dd4b744b5a271e6fffb85e20d" is_pre_election: true
I20260807 15:01:46.889376 2573 raft_consensus.cc:2454] T 6dc628ba12524bd984c40f5887a2a305 P a283eabbf21d487b805495f923dc8245 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0955c4f88cf748acbafe7c0a8c44c745 in term 1.
I20260807 15:01:46.889488 2288 raft_consensus.cc:2454] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0955c4f88cf748acbafe7c0a8c44c745 in term 1.
I20260807 15:01:46.889619 2362 leader_election.cc:302] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0955c4f88cf748acbafe7c0a8c44c745, a283eabbf21d487b805495f923dc8245; no voters:
I20260807 15:01:46.889719 2647 raft_consensus.cc:2781] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:46.889935 2647 raft_consensus.cc:686] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [term 1 LEADER]: Becoming Leader. State: Replica: 0955c4f88cf748acbafe7c0a8c44c745, State: Running, Role: LEADER
I20260807 15:01:46.890152 2647 consensus_queue.cc:237] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } }
I20260807 15:01:46.891361 2137 catalog_manager.cc:5495] T 6dc628ba12524bd984c40f5887a2a305 P 0955c4f88cf748acbafe7c0a8c44c745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0955c4f88cf748acbafe7c0a8c44c745 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "0955c4f88cf748acbafe7c0a8c44c745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a283eabbf21d487b805495f923dc8245" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b0448dd4b744b5a271e6fffb85e20d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0955c4f88cf748acbafe7c0a8c44c745" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 46341 } health_report { overall_health: HEALTHY } } }
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:46.911870 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.903104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.1.127.193:39129)
0807 15:01:46.903218 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.903220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.903334 (+ 114us) server_negotiation.cc:407] Connection header received
0807 15:01:46.903435 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.903436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.903448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.903503 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.904120 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.904640 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.905414 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.905633 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.905713 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.908549 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.908567 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.908567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.908585 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.910390 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.910919 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.910924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.910924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.910998 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.911224 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.911225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.911226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.911400 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.911619 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.911661 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.911754 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:46.940478 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.932008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.1.127.193:39129)
0807 15:01:46.932096 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.932098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.932205 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:01:46.932307 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.932308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.932320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.932396 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.933069 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.933599 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.934380 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.934615 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.934727 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.937312 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.937313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.937314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.937322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.939034 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.939442 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.939443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.939444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.939508 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.939833 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.939835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.939835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.939968 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.940076 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.940317 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.940403 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 15:01:46.961642 2338 tablet.cc:2369] T 6dc628ba12524bd984c40f5887a2a305 P 86b0448dd4b744b5a271e6fffb85e20d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:01:46 dist-test-slave-8tdl krb5kdc[2090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:46.970533 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:46.961645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.1.127.193:39129)
0807 15:01:46.961739 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:46.961741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:46.961913 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:01:46.962086 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:46.962087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:46.962098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:46.962149 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:46.962847 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.963355 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.964169 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:46.964350 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:46.964404 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:46.967367 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:46.967368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:46.967368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:46.967376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:46.969068 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.969500 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.969501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.969502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.969569 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:46.969852 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:46.969853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:46.969854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:46.969990 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:46.970086 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:46.970288 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:01:46.970375 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 15:01:46.971622 2246 tablet_service.cc:2908] Rejecting scan request for tablet 6dc628ba12524bd984c40f5887a2a305: Uninitialized: clean time has not yet been initialized
I20260807 15:01:46.973448 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2202
I20260807 15:01:46.980744 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2342
I20260807 15:01:46.987777 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2486
I20260807 15:01:46.993883 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2105
2026-08-07T15:01:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2136 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:47 dist-test-slave-8tdl krb5kdc[2665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:47 dist-test-slave-8tdl krb5kdc[2665](info): set up 2 sockets
Aug 07 15:01:47 dist-test-slave-8tdl krb5kdc[2665](info): commencing operation
krb5kdc: starting...
W20260807 15:01:48.086627 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:48.223194 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39097
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:39097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:48.309216 2678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:48.309484 2678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:48.309523 2678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:48.310794 2678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:01:48.310874 2678 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:01:48.310896 2678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:48.310920 2678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:48.310964 2678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:01:48.310989 2678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:01:48.312623 2678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:39097
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:48.312937 2678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:48.313206 2678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:48.318629 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:48.318809 2678 server_base.cc:1034] running on GCE node
W20260807 15:01:48.318636 2684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.318657 2686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:48.319345 2678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:48.319842 2678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:48.321049 2678 hybrid_clock.cc:648] HybridClock initialized: now 1786114908321030 us; error 32 us; skew 500 ppm
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:48.329444 2678 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:01:48.330011 2678 webserver.cc:460] Webserver started at http://127.1.127.254:40651/ using document root <none> and password file <none>
I20260807 15:01:48.330286 2678 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:48.330358 2678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:48.330524 2678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:48.331804 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "53c7fd63c2fc42e889aa6bad378d9aed"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "88a9340c74cbb133ef82a375688450db"
server_key_iv: "14aaf8750cfb179c4c26be23f3188d7d"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.332206 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "53c7fd63c2fc42e889aa6bad378d9aed"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "88a9340c74cbb133ef82a375688450db"
server_key_iv: "14aaf8750cfb179c4c26be23f3188d7d"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.333796 2678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:48.334647 2693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.334890 2678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:48.335043 2678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "53c7fd63c2fc42e889aa6bad378d9aed"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "88a9340c74cbb133ef82a375688450db"
server_key_iv: "14aaf8750cfb179c4c26be23f3188d7d"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.335131 2678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:48.356108 2678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:48.358224 2678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:48.358417 2678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:48.363967 2678 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:39097
I20260807 15:01:48.364029 2761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:39097 every 8 connection(s)
I20260807 15:01:48.364393 2678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:01:48.365226 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:48.367686 2763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Bootstrap starting.
I20260807 15:01:48.368343 2763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:48.368654 2763 log.cc:824] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:48.369484 2763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: No bootstrap required, opened a new log
I20260807 15:01:48.369786 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2678
I20260807 15:01:48.369870 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:01:48.370090 1535 external_mini_cluster.cc:1428] Setting key a2831e265ee19b19c5a8895f42ae7af1
I20260807 15:01:48.372555 2763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } }
I20260807 15:01:48.372748 2763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:48.372788 2763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c7fd63c2fc42e889aa6bad378d9aed, State: Initialized, Role: FOLLOWER
I20260807 15:01:48.372965 2763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } }
I20260807 15:01:48.373041 2763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:01:48.373064 2763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:01:48.373134 2763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:48.374014 2763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } }
I20260807 15:01:48.374131 2763 leader_election.cc:302] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53c7fd63c2fc42e889aa6bad378d9aed; no voters:
I20260807 15:01:48.374310 2763 leader_election.cc:288] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:01:48.374405 2768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:48.374601 2768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [term 1 LEADER]: Becoming Leader. State: Replica: 53c7fd63c2fc42e889aa6bad378d9aed, State: Running, Role: LEADER
I20260807 15:01:48.374639 2763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:01:48.376155 2768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } }
I20260807 15:01:48.377049 2770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } } }
I20260807 15:01:48.377048 2769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53c7fd63c2fc42e889aa6bad378d9aed. Latest consensus state: current_term: 1 leader_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53c7fd63c2fc42e889aa6bad378d9aed" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39097 } } }
I20260807 15:01:48.377620 2770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:48.377640 2769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed [sys.catalog]: This master's current role is: LEADER
W20260807 15:01:48.377458 2782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:01:48.377822 2782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:01:48.377923 2785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:01:48.378165 2785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:48.380254 2785 catalog_manager.cc:1347] Generated new cluster ID: 56023dac8e8a43c8aad210c28cfa644b
I20260807 15:01:48.380326 2785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:01:48.384176 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.370651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.1.127.254:39097)
0807 15:01:48.370909 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.370920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.370954 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:01:48.372516 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.372526 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.372602 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.372770 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.373896 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.375354 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.376277 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.376878 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.376993 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.380209 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.380213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.380213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.380227 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.382306 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.382836 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.382839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.382840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.382971 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.383207 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.383208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.383208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.383550 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.383712 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.383826 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.383966 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":122,"threads_started":1}
I20260807 15:01:48.392246 2785 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:01:48.392891 2785 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:01:48.399215 2785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Generated new TSK 0
I20260807 15:01:48.399513 2785 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:01:48.432472 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39097
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:48.523097 2789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:48.523361 2789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:48.523428 2789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:48.524801 2789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:48.524881 2789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:48.524916 2789 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:48.524956 2789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:01:48.526544 2789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:48.526860 2789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:48.527118 2789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:48.532594 2795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.532653 2797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.532689 2794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:48.532909 2789 server_base.cc:1034] running on GCE node
I20260807 15:01:48.533406 2789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:48.533639 2789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:48.534760 2789 hybrid_clock.cc:648] HybridClock initialized: now 1786114908534752 us; error 40 us; skew 500 ppm
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:48.537597 2789 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:01:48.538158 2789 webserver.cc:460] Webserver started at http://127.1.127.193:39969/ using document root <none> and password file <none>
I20260807 15:01:48.538405 2789 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:48.538475 2789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:48.538640 2789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:48.539892 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "b2a22b367bc247d6bf940b99f40045ef"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "af875033816b608da9b1a5d409506894"
server_key_iv: "690308fb574036db70592a7488371508"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.540277 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "b2a22b367bc247d6bf940b99f40045ef"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "af875033816b608da9b1a5d409506894"
server_key_iv: "690308fb574036db70592a7488371508"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.541822 2789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:01:48.542817 2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.543011 2789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:48.543160 2789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "b2a22b367bc247d6bf940b99f40045ef"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "af875033816b608da9b1a5d409506894"
server_key_iv: "690308fb574036db70592a7488371508"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.543246 2789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:48.569459 2789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:48.572098 2789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:48.572285 2789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:48.572867 2789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:48.572930 2789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.572979 2789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:48.573007 2789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:48.583271 2789 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:40333
I20260807 15:01:48.583405 2925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:40333 every 8 connection(s)
I20260807 15:01:48.583757 2789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:01:48.586257 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.573758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:57911 (local address 127.1.127.254:39097)
0807 15:01:48.573847 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.573850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.574146 (+ 296us) server_negotiation.cc:407] Connection header received
0807 15:01:48.574729 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.574731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.574756 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.574883 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.575937 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.577013 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.578812 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.579170 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.579246 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.582846 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.582848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.582848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.582857 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.584839 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.585317 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.585318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.585319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.585380 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.585613 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.585613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.585614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.585735 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.585790 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.586107 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.586187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:01:48.586333 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.573739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.193:57911)
0807 15:01:48.573932 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:48.573976 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:48.574544 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:48.575049 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:48.575053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:48.575496 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:48.575728 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.575732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.577130 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.577132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.578621 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.578624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.579253 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.579254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.579412 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.580110 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:48.580121 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:48.582729 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:48.584953 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.584961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.584964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.585237 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.585445 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.585445 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.585446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.585552 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.585794 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:48.585798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:48.585922 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 15:01:48.586008 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:01:48.586138 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260807 15:01:48.587064 2926 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39097
I20260807 15:01:48.587186 2926 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:48.587401 2926 heartbeater.cc:511] Master 127.1.127.254:39097 requested a full tablet report, sending...
I20260807 15:01:48.588079 2711 ts_manager.cc:194] Registered new tserver with Master: b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333)
I20260807 15:01:48.588958 2711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:57911
I20260807 15:01:48.589480 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2789
I20260807 15:01:48.589643 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:01:48.589852 1535 external_mini_cluster.cc:1428] Setting key 85ad7a19ab414aa7839b8ffe237a42be
I20260807 15:01:48.595269 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.588503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:53469 (local address 127.1.127.254:39097)
0807 15:01:48.588545 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.588548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.588695 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:01:48.588773 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.588775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.588794 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.588852 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.589218 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.589905 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.590980 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.591204 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.591284 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.592345 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.592347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.592347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.592357 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.593962 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.594424 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.594425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.594426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.594482 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.594736 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.594737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.594737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.594882 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.594943 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.595135 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.595204 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:01:48.595399 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.588438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.193:53469)
0807 15:01:48.588612 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:48.588631 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:48.588712 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:48.588857 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:48.588858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:48.589059 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:48.589153 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.589155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.590025 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.590026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.590873 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.590876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.591283 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.591284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.591379 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.591883 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:48.591889 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:48.592215 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:48.594069 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.594071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.594071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.594323 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.594554 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.594555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.594555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.594632 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.594963 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:48.594965 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:48.595026 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 15:01:48.595119 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:01:48.595249 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:01:48.621564 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39097
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:48.708683 2935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:48.708914 2935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:48.708940 2935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:48.710094 2935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:48.710134 2935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:48.710155 2935 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:48.710173 2935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:01:48.711735 2935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:48.711957 2935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:48.712149 2935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:48.717797 2941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.717803 2943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.717801 2940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:48.717978 2935 server_base.cc:1034] running on GCE node
I20260807 15:01:48.718357 2935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:48.718600 2935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:48.719797 2935 hybrid_clock.cc:648] HybridClock initialized: now 1786114908719781 us; error 31 us; skew 500 ppm
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:48.722824 2935 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:01:48.723416 2935 webserver.cc:460] Webserver started at http://127.1.127.194:37283/ using document root <none> and password file <none>
I20260807 15:01:48.723685 2935 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:48.723760 2935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:48.723912 2935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:48.725139 2935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "5516ac65ffca43e0ae322adc65c9b53b"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "4f938fd513c372d214183625deff8f8d"
server_key_iv: "d79b0ba38e4a989f4cddfd970e4547eb"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.725536 2935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "5516ac65ffca43e0ae322adc65c9b53b"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "4f938fd513c372d214183625deff8f8d"
server_key_iv: "d79b0ba38e4a989f4cddfd970e4547eb"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.727162 2935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:48.728191 2950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.728503 2935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:48.728646 2935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "5516ac65ffca43e0ae322adc65c9b53b"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "4f938fd513c372d214183625deff8f8d"
server_key_iv: "d79b0ba38e4a989f4cddfd970e4547eb"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.728734 2935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:48.783188 2935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:48.785691 2935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:48.785883 2935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:48.786464 2935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:48.786547 2935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.786595 2935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:48.786628 2935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:48.794927 2935 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:39377
I20260807 15:01:48.795037 3071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:39377 every 8 connection(s)
I20260807 15:01:48.795389 2935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:01:48.798210 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.787115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:34347 (local address 127.1.127.254:39097)
0807 15:01:48.787177 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.787179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.787453 (+ 274us) server_negotiation.cc:407] Connection header received
0807 15:01:48.788234 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.788235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.788249 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.788327 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.789350 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.790206 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.791548 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.791770 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.791983 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.794693 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.794696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.794697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.794705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.796464 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.796955 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.796957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.796957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.797024 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.797345 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.797346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.797346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.797533 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.797603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.798022 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.798123 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:01:48.798233 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.787064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.194:34347)
0807 15:01:48.787257 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:48.787337 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:48.788113 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:48.788518 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:48.788522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:48.788883 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:48.789224 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.789233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.790427 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.790429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.791427 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.791430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.792022 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.792023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.792172 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.792801 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:48.792813 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:48.794544 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:48.796566 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.796574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.796576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.796854 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.797128 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.797130 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.797130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.797250 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.797628 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:48.797635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:48.797785 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 15:01:48.797893 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 15:01:48.798059 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260807 15:01:48.798846 3072 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39097
I20260807 15:01:48.798960 3072 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:48.799033 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 2935
I20260807 15:01:48.799149 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:01:48.799189 3072 heartbeater.cc:511] Master 127.1.127.254:39097 requested a full tablet report, sending...
I20260807 15:01:48.799333 1535 external_mini_cluster.cc:1428] Setting key 65b9a5ff39e958f83e321c0ff4d5a5a7
I20260807 15:01:48.799661 2712 ts_manager.cc:194] Registered new tserver with Master: 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194:39377)
I20260807 15:01:48.800179 2712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:34347
I20260807 15:01:48.807226 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.800581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:40615 (local address 127.1.127.254:39097)
0807 15:01:48.800741 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.800743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.800751 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:01:48.800771 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.800772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.800783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.800848 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.801304 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.802043 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.803171 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.803367 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.803454 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.804469 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.804470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.804471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.804478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.805883 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.806287 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.806288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.806288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.806349 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.806561 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.806562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.806562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.806792 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.806871 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.807047 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.807124 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260807 15:01:48.807245 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.800125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.194:40615)
0807 15:01:48.800272 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:48.800285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:48.800335 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:48.800911 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:48.800912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:48.801131 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:48.801210 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.801212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.802176 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.802177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.803091 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.803094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.803457 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.803458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.803538 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.804038 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:48.804044 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:48.804370 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:48.806035 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.806036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.806037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.806207 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.806423 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.806424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.806424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.806487 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.806861 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:48.806862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:48.806922 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:01:48.807005 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:01:48.807125 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:01:48.835261 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39097
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:48.924019 3081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:48.924309 3081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:48.924356 3081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:48.925902 3081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:48.925987 3081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:48.926021 3081 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:48.926043 3081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:01:48.927819 3081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:48.928067 3081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:48.928265 3081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:01:48.934721 3081 server_base.cc:1034] running on GCE node
W20260807 15:01:48.934644 3087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.934659 3086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:48.934644 3089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:48.935178 3081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:48.935422 3081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:48.936605 3081 hybrid_clock.cc:648] HybridClock initialized: now 1786114908936582 us; error 33 us; skew 500 ppm
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:48.940028 3081 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:01:48.940495 3081 webserver.cc:460] Webserver started at http://127.1.127.195:37327/ using document root <none> and password file <none>
I20260807 15:01:48.940773 3081 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:48.940864 3081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:48.941031 3081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:48.942191 3081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "77e0eba8ffeb4cfab62744eef6f29a65"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "d884bf1c3a549ed01f28538dbed0e110"
server_key_iv: "a465d034907b0a876369ccd854152364"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.942606 3081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "77e0eba8ffeb4cfab62744eef6f29a65"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "d884bf1c3a549ed01f28538dbed0e110"
server_key_iv: "a465d034907b0a876369ccd854152364"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.944257 3081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:48.945710 3096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.945999 3081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:48.946178 3081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "77e0eba8ffeb4cfab62744eef6f29a65"
format_stamp: "Formatted at 2026-08-07 15:01:48 on dist-test-slave-8tdl"
server_key: "d884bf1c3a549ed01f28538dbed0e110"
server_key_iv: "a465d034907b0a876369ccd854152364"
server_key_version: "encryptionkey@0"
I20260807 15:01:48.946281 3081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:48.983385 3081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:48.985898 3081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:48.986100 3081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:48.986769 3081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:48.986873 3081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:48.986922 3081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:48.986948 3081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:01:48 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:48.995317 3081 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:41919
I20260807 15:01:48.995684 3217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:41919 every 8 connection(s)
I20260807 15:01:48.995734 3081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:01:48.998199 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.987558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:49091 (local address 127.1.127.254:39097)
0807 15:01:48.987646 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:48.987649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:48.987855 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:01:48.988527 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:48.988529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:48.988545 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:48.988630 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:48.989682 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.990645 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.992235 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.992475 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.992565 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.995050 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:48.995054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:48.995054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:48.995064 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:48.996693 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.997147 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.997148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.997149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.997215 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:48.997479 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:48.997480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:48.997480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:48.997610 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:48.997683 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:48.998012 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:01:48.998107 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:01:48.998385 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:48.987476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.195:49091)
0807 15:01:48.987691 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:48.987736 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:48.988407 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:48.988710 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:48.988713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:48.989221 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:48.989561 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.989569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.990814 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.990815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.992123 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:48.992126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:48.992590 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:48.992591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:48.992725 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:48.993347 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:48.993356 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:48.994918 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:48.996798 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.996807 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.996810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.997056 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.997285 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:48.997286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:48.997286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:48.997402 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:48.997686 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:48.997693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:48.997858 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 15:01:48.997966 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 15:01:48.998141 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260807 15:01:48.999042 3218 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39097
I20260807 15:01:48.999163 3218 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:48.999382 3218 heartbeater.cc:511] Master 127.1.127.254:39097 requested a full tablet report, sending...
I20260807 15:01:49.000005 2711 ts_manager.cc:194] Registered new tserver with Master: 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919)
I20260807 15:01:49.000535 2711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:49091
I20260807 15:01:49.002137 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 3081
I20260807 15:01:49.002259 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:01:49.002424 1535 external_mini_cluster.cc:1428] Setting key f2ae9536107eb4fa350279a794facb3a
I20260807 15:01:49.003755 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:01:49.007535 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.000206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.1.127.195:52663)
0807 15:01:49.000383 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.000394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.000484 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.000653 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.000656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.000858 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:49.000963 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.000965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.001667 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.001668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.002174 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.002176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.002710 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.002711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.002797 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.003237 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:49.003241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:49.003659 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:49.005832 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:49.005833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:49.005833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:49.006215 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:49.006504 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:49.006505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:49.006506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:49.006572 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:49.007191 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:49.007192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:49.007250 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.007326 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.007438 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260807 15:01:49.007591 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.000159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:52663 (local address 127.1.127.254:39097)
0807 15:01:49.000211 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.000213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.000468 (+ 255us) server_negotiation.cc:407] Connection header received
0807 15:01:49.000547 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.000548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.000559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.000613 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:49.001055 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.001521 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.002331 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.002588 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.002708 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.003762 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:49.003763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:49.003763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:49.003771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:49.005713 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.006334 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.006336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.006336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.006416 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.006657 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.006659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.006659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.006963 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:49.007192 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.007378 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.007488 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 15:01:49 dist-test-slave-8tdl krb5kdc[2665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:01:49 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:49.033087 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.024688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.1.127.254:39097)
0807 15:01:49.024758 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.024761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.024771 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:49.024886 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.024887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.024910 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.024986 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:49.025644 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.026292 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.027241 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.027499 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.027574 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.030218 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:49.030219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:49.030220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:49.030228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:49.031850 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.032228 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.032229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.032229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.032336 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.032550 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.032551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.032551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.032714 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:49.032805 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.032930 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.033008 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:01:49.035360 2711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:49.037071 2711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:49.044472 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.040021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.1.127.194:39377)
0807 15:01:49.040237 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.040242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.040507 (+ 265us) server_negotiation.cc:407] Connection header received
0807 15:01:49.040625 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.040628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.040647 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.040738 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.041045 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.042198 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.043475 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.043986 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044072 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044132 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.044306 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.044349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260807 15:01:49.044502 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.039871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39377 (local address 127.0.0.1:39128)
0807 15:01:49.040332 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.040400 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.040537 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.040850 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.040854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.040859 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.040943 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.040946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.042298 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.042299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.043380 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.043383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.044086 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.044190 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044202 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.044289 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.044356 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":29,"thread_start_us":143,"threads_started":1}
I20260807 15:01:49.044750 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.040309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.1.127.195:41919)
0807 15:01:49.040534 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.040539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.040755 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:01:49.040806 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.040808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.040825 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.040900 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.041075 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.042210 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.043523 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.044243 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044307 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044357 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.044590 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.044638 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260807 15:01:49.044775 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.040195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41919 (local address 127.0.0.1:49306)
0807 15:01:49.040640 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.040649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.040692 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.040897 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.040898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.040900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.040963 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.040965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.042309 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.042310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.043421 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.043424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.044392 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.044453 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044456 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.044505 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.044547 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":82,"threads_started":1}
I20260807 15:01:49.045025 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.040035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.1.127.193:40333)
0807 15:01:49.040632 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.040637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.040648 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:01:49.040690 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.040692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.040708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.040777 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.041032 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.041761 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.043216 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.043988 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044061 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044125 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.044719 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.044772 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":92,"threads_started":1}
I20260807 15:01:49.044492 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.039870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40333 (local address 127.0.0.1:36694)
0807 15:01:49.040332 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.040393 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.040472 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.040850 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.040854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.040856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.040943 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.040946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.041860 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.041861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.043119 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.043122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.044164 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.044165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.044244 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.044249 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.044299 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.044358 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":6,"thread_start_us":110,"threads_started":1}
I20260807 15:01:49.045625 3146 tablet_service.cc:1467] Processing CreateTablet for tablet 08a9da25bf974f2db962ac9115374cee (DEFAULT_TABLE table=test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.045719 2857 tablet_service.cc:1467] Processing CreateTablet for tablet 08a9da25bf974f2db962ac9115374cee (DEFAULT_TABLE table=test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.046056 3146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a9da25bf974f2db962ac9115374cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.046064 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a9da25bf974f2db962ac9115374cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.047369 3003 tablet_service.cc:1467] Processing CreateTablet for tablet 08a9da25bf974f2db962ac9115374cee (DEFAULT_TABLE table=test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.047791 3003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a9da25bf974f2db962ac9115374cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.049280 3241 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Bootstrap starting.
I20260807 15:01:49.049760 3242 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Bootstrap starting.
I20260807 15:01:49.050724 3241 tablet_bootstrap.cc:654] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.050825 3242 tablet_bootstrap.cc:654] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.051229 3242 log.cc:824] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:49.051236 3241 log.cc:824] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:49.051961 3245 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Bootstrap starting.
I20260807 15:01:49.052151 3242 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: No bootstrap required, opened a new log
I20260807 15:01:49.052227 3242 ts_tablet_manager.cc:1397] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:01:49.052466 3241 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: No bootstrap required, opened a new log
I20260807 15:01:49.052515 3241 ts_tablet_manager.cc:1397] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:01:49.052647 3245 tablet_bootstrap.cc:654] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.052990 3245 log.cc:824] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:49.053810 3245 tablet_bootstrap.cc:492] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: No bootstrap required, opened a new log
I20260807 15:01:49.053871 3245 ts_tablet_manager.cc:1397] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:49.054262 3241 raft_consensus.cc:348] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.054392 3241 raft_consensus.cc:374] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.054414 3241 raft_consensus.cc:729] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a22b367bc247d6bf940b99f40045ef, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.054538 3241 consensus_queue.cc:260] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.054538 3242 raft_consensus.cc:348] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.054734 3242 raft_consensus.cc:374] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.054772 3242 raft_consensus.cc:729] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e0eba8ffeb4cfab62744eef6f29a65, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.054883 3241 ts_tablet_manager.cc:1428] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:01:49.054927 3242 consensus_queue.cc:260] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.055019 2926 heartbeater.cc:503] Master 127.1.127.254:39097 was elected leader, sending a full tablet report...
I20260807 15:01:49.055202 3242 ts_tablet_manager.cc:1428] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 15:01:49.055312 3218 heartbeater.cc:503] Master 127.1.127.254:39097 was elected leader, sending a full tablet report...
I20260807 15:01:49.056041 3245 raft_consensus.cc:348] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.056188 3245 raft_consensus.cc:374] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.056211 3245 raft_consensus.cc:729] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5516ac65ffca43e0ae322adc65c9b53b, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.056327 3245 consensus_queue.cc:260] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.056609 3245 ts_tablet_manager.cc:1428] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:01:49.056703 3072 heartbeater.cc:503] Master 127.1.127.254:39097 was elected leader, sending a full tablet report...
W20260807 15:01:49.084659 2927 tablet.cc:2369] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:49.162163 3249 raft_consensus.cc:482] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:49.162340 3249 raft_consensus.cc:504] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.162765 3249 leader_election.cc:288] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919)
I20260807 15:01:49.167330 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.163379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:48687 (local address 127.1.127.195:41919)
0807 15:01:49.163463 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.163466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.163475 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:01:49.163531 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.163532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.163540 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.163582 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.163755 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.164526 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166149 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.166820 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166917 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.166927 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.167204 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.167254 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:01:49.167330 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.163183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:33611 (local address 127.1.127.193:40333)
0807 15:01:49.163290 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.163292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.163304 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:01:49.163338 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.163339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.163348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.163459 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.163657 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.164486 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166150 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.166820 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166917 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.166927 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.167205 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.167253 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:01:49.167309 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.163063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40333 (local address 127.1.127.194:33611)
0807 15:01:49.163136 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.163150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.163199 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.163467 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.163467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.163469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.163553 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.163555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.164638 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.164639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.166021 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.166023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.166953 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.167074 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.167082 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.167149 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.167203 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":121}
I20260807 15:01:49.167320 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.163214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41919 (local address 127.1.127.194:48687)
0807 15:01:49.163398 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.163408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.163467 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.163611 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.163612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.163614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.163676 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.163678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.164638 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.164639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.166005 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.166009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.166953 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.166954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.167074 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.167082 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.167149 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.167203 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":44,"thread_start_us":117,"threads_started":1}
I20260807 15:01:49.167820 3167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a9da25bf974f2db962ac9115374cee" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" is_pre_election: true
I20260807 15:01:49.167816 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a9da25bf974f2db962ac9115374cee" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef" is_pre_election: true
I20260807 15:01:49.168008 3167 raft_consensus.cc:2454] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 0.
I20260807 15:01:49.168036 2869 raft_consensus.cc:2454] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 0.
I20260807 15:01:49.168546 2952 leader_election.cc:302] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5516ac65ffca43e0ae322adc65c9b53b, b2a22b367bc247d6bf940b99f40045ef; no voters:
I20260807 15:01:49.168735 3249 raft_consensus.cc:2781] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:49.168929 3249 raft_consensus.cc:482] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:49.168975 3249 raft_consensus.cc:3037] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.170107 3249 raft_consensus.cc:504] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.170295 3249 leader_election.cc:288] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 election: Requested vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919)
I20260807 15:01:49.170589 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a9da25bf974f2db962ac9115374cee" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef"
I20260807 15:01:49.170589 3167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a9da25bf974f2db962ac9115374cee" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e0eba8ffeb4cfab62744eef6f29a65"
I20260807 15:01:49.170730 3167 raft_consensus.cc:3037] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.170728 2869 raft_consensus.cc:3037] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.171720 3167 raft_consensus.cc:2454] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 1.
I20260807 15:01:49.171720 2869 raft_consensus.cc:2454] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 1.
I20260807 15:01:49.171962 2952 leader_election.cc:302] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5516ac65ffca43e0ae322adc65c9b53b, 77e0eba8ffeb4cfab62744eef6f29a65; no voters:
I20260807 15:01:49.172070 3249 raft_consensus.cc:2781] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:49.172268 3249 raft_consensus.cc:686] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 LEADER]: Becoming Leader. State: Replica: 5516ac65ffca43e0ae322adc65c9b53b, State: Running, Role: LEADER
I20260807 15:01:49.172435 3249 consensus_queue.cc:237] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.173720 2709 catalog_manager.cc:5495] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194). New cstate: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } health_report { overall_health: UNKNOWN } } }
I20260807 15:01:49.195457 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.192554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.1.127.194:39377)
0807 15:01:49.192722 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.192724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.192733 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:01:49.192772 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.192773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.192792 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.192851 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:49.193093 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.193636 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.194493 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.194801 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.194880 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.195124 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.195231 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.195299 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.195343 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.195364 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 15:01:49.197852 2869 raft_consensus.cc:1261] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Refusing update from remote peer 5516ac65ffca43e0ae322adc65c9b53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.198321 3253 consensus_queue.cc:1035] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.199623 3167 raft_consensus.cc:1261] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Refusing update from remote peer 5516ac65ffca43e0ae322adc65c9b53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.199926 3249 consensus_queue.cc:1035] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.201414 3257 mvcc.cc:205] Tried to move back new op lower bound from 7315926668070719488 to 7315926667971149824. Current Snapshot: MvccSnapshot[applied={T|T < 7315926668070719488}]
I20260807 15:01:49.206354 2709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:49.206502 2709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:49.207592 2709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Sending DeleteTablet for 3 replicas of tablet 08a9da25bf974f2db962ac9115374cee
I20260807 15:01:49.208175 2857 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a9da25bf974f2db962ac9115374cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260807 15:01:49.208196 3003 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a9da25bf974f2db962ac9115374cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39128
I20260807 15:01:49.208261 2709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:49.208415 2709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:49.208448 3146 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a9da25bf974f2db962ac9115374cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:49306
I20260807 15:01:49.208666 3267 tablet_replica.cc:330] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: stopping tablet replica
I20260807 15:01:49.208782 3267 raft_consensus.cc:2229] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:49.208909 3267 raft_consensus.cc:2258] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.209432 3266 tablet_replica.cc:330] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: stopping tablet replica
I20260807 15:01:49.209885 3003 tablet_service.cc:1467] Processing CreateTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee (DEFAULT_TABLE table=test-table [id=14cd5f490d3d443580861f4a9dbfc750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.209959 2857 tablet_service.cc:1467] Processing CreateTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee (DEFAULT_TABLE table=test-table [id=14cd5f490d3d443580861f4a9dbfc750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.209972 3146 tablet_service.cc:1467] Processing CreateTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee (DEFAULT_TABLE table=test-table [id=14cd5f490d3d443580861f4a9dbfc750]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:49.210043 3003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ddd224fe16c4bf5986d267470ec59ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.210078 3146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ddd224fe16c4bf5986d267470ec59ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.210100 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ddd224fe16c4bf5986d267470ec59ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.211540 3245 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Bootstrap starting.
I20260807 15:01:49.211607 3242 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Bootstrap starting.
I20260807 15:01:49.212000 3245 tablet_bootstrap.cc:654] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.212328 3242 tablet_bootstrap.cc:654] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.212704 3241 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Bootstrap starting.
I20260807 15:01:49.212834 3245 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: No bootstrap required, opened a new log
I20260807 15:01:49.212880 3245 ts_tablet_manager.cc:1397] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:49.213020 3245 raft_consensus.cc:348] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.213091 3245 raft_consensus.cc:374] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.213110 3245 raft_consensus.cc:729] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5516ac65ffca43e0ae322adc65c9b53b, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.213105 3241 tablet_bootstrap.cc:654] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.213140 3267 ts_tablet_manager.cc:1905] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.213160 3245 consensus_queue.cc:260] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.213244 3245 ts_tablet_manager.cc:1428] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:01:49.214612 3265 tablet_replica.cc:330] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: stopping tablet replica
I20260807 15:01:49.209569 3266 raft_consensus.cc:2229] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:49.214779 3265 raft_consensus.cc:2229] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:49.214926 3266 raft_consensus.cc:2258] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.215026 3267 ts_tablet_manager.cc:1918] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:01:49.215166 3267 log.cc:1196] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/08a9da25bf974f2db962ac9115374cee
I20260807 15:01:49.215479 3266 ts_tablet_manager.cc:1905] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.215564 3267 ts_tablet_manager.cc:1939] T 08a9da25bf974f2db962ac9115374cee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting consensus metadata
I20260807 15:01:49.215603 3242 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: No bootstrap required, opened a new log
I20260807 15:01:49.215674 3242 ts_tablet_manager.cc:1397] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 15:01:49.215806 3241 tablet_bootstrap.cc:492] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: No bootstrap required, opened a new log
I20260807 15:01:49.215847 3241 ts_tablet_manager.cc:1397] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260807 15:01:49.215826 3242 raft_consensus.cc:348] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.215896 3242 raft_consensus.cc:374] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.215920 3242 raft_consensus.cc:729] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e0eba8ffeb4cfab62744eef6f29a65, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.215967 3242 consensus_queue.cc:260] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.215998 3241 raft_consensus.cc:348] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.216048 3242 ts_tablet_manager.cc:1428] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:49.216070 3241 raft_consensus.cc:374] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.216089 3241 raft_consensus.cc:729] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a22b367bc247d6bf940b99f40045ef, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.216131 3241 consensus_queue.cc:260] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.216228 3241 ts_tablet_manager.cc:1428] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:49.217447 2695 catalog_manager.cc:4841] TS 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919): tablet 08a9da25bf974f2db962ac9115374cee (table test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]) successfully deleted
I20260807 15:01:49.214956 3265 raft_consensus.cc:2258] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.217962 3266 ts_tablet_manager.cc:1918] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:01:49.218087 3266 log.cc:1196] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/08a9da25bf974f2db962ac9115374cee
I20260807 15:01:49.218340 3265 ts_tablet_manager.cc:1905] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.218502 3266 ts_tablet_manager.cc:1939] T 08a9da25bf974f2db962ac9115374cee P b2a22b367bc247d6bf940b99f40045ef: Deleting consensus metadata
I20260807 15:01:49.218994 2695 catalog_manager.cc:4841] TS b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333): tablet 08a9da25bf974f2db962ac9115374cee (table test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]) successfully deleted
I20260807 15:01:49.220366 3265 ts_tablet_manager.cc:1918] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:01:49.220444 3265 log.cc:1196] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/08a9da25bf974f2db962ac9115374cee
I20260807 15:01:49.220652 3265 ts_tablet_manager.cc:1939] T 08a9da25bf974f2db962ac9115374cee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting consensus metadata
I20260807 15:01:49.221127 2697 catalog_manager.cc:4841] TS 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194:39377): tablet 08a9da25bf974f2db962ac9115374cee (table test-table [id=f8bf41df4cc745f491e1c4d36c5fabdf]) successfully deleted
W20260807 15:01:49.246495 3219 tablet.cc:2369] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:01:49.296326 3073 tablet.cc:2369] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:49.414018 3248 raft_consensus.cc:482] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:49.414134 3248 raft_consensus.cc:504] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.414563 3248 leader_election.cc:288] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194:39377)
I20260807 15:01:49.417789 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.414863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:51635 (local address 127.1.127.193:40333)
0807 15:01:49.414922 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.414924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.415055 (+ 131us) server_negotiation.cc:407] Connection header received
0807 15:01:49.415135 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.415136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.415146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.415222 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.415386 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.415882 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.416954 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.417381 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.417473 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.417484 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.417664 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.417707 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:01:49.417815 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.414954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:47357 (local address 127.1.127.194:39377)
0807 15:01:49.415013 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.415015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.415137 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:01:49.415212 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.415213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.415225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.415302 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:49.415484 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.416014 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.417015 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.417400 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.417488 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.417499 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.417689 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.417734 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:01:49.417810 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.414777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40333 (local address 127.1.127.195:51635)
0807 15:01:49.414856 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.414981 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.415071 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.415224 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.415225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.415226 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.415308 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.415310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.415977 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.415978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.416832 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.416834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.417472 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.417473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.417569 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.417572 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.417651 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.417699 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":28}
I20260807 15:01:49.418112 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.414923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39377 (local address 127.1.127.195:47357)
0807 15:01:49.415056 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.415064 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.415155 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.415307 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.415308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.415310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:49.415387 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.415389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.416106 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.416107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.416896 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.416898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.417516 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.417517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.417595 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.417598 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.417678 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.417722 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260807 15:01:49.418184 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ddd224fe16c4bf5986d267470ec59ee" candidate_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef" is_pre_election: true
I20260807 15:01:49.418326 2869 raft_consensus.cc:2454] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e0eba8ffeb4cfab62744eef6f29a65 in term 0.
I20260807 15:01:49.418527 3098 leader_election.cc:302] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e0eba8ffeb4cfab62744eef6f29a65, b2a22b367bc247d6bf940b99f40045ef; no voters:
I20260807 15:01:49.418673 3248 raft_consensus.cc:2781] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:49.418737 3248 raft_consensus.cc:482] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:49.418771 3248 raft_consensus.cc:3037] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.418668 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ddd224fe16c4bf5986d267470ec59ee" candidate_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5516ac65ffca43e0ae322adc65c9b53b" is_pre_election: true
I20260807 15:01:49.418857 3024 raft_consensus.cc:2454] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e0eba8ffeb4cfab62744eef6f29a65 in term 0.
I20260807 15:01:49.419548 3248 raft_consensus.cc:504] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.419719 3248 leader_election.cc:288] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [CANDIDATE]: Term 1 election: Requested vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194:39377)
I20260807 15:01:49.419914 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ddd224fe16c4bf5986d267470ec59ee" candidate_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef"
I20260807 15:01:49.419986 2869 raft_consensus.cc:3037] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.419917 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ddd224fe16c4bf5986d267470ec59ee" candidate_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5516ac65ffca43e0ae322adc65c9b53b"
I20260807 15:01:49.420032 3024 raft_consensus.cc:3037] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.420717 2869 raft_consensus.cc:2454] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e0eba8ffeb4cfab62744eef6f29a65 in term 1.
I20260807 15:01:49.420851 3024 raft_consensus.cc:2454] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e0eba8ffeb4cfab62744eef6f29a65 in term 1.
I20260807 15:01:49.420903 3098 leader_election.cc:302] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e0eba8ffeb4cfab62744eef6f29a65, b2a22b367bc247d6bf940b99f40045ef; no voters:
I20260807 15:01:49.421005 3248 raft_consensus.cc:2781] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:49.421186 3248 raft_consensus.cc:686] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 LEADER]: Becoming Leader. State: Replica: 77e0eba8ffeb4cfab62744eef6f29a65, State: Running, Role: LEADER
I20260807 15:01:49.421254 3248 consensus_queue.cc:237] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } }
I20260807 15:01:49.422367 2712 catalog_manager.cc:5495] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } health_report { overall_health: HEALTHY } } }
Aug 07 15:01:49 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114908, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:49.484215 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.475941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39097 (local address 127.0.0.1:49632)
0807 15:01:49.476197 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.476208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.476496 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.476670 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.476671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.477003 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:49.477062 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.477064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.477749 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.477750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.478277 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.478279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.478772 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.478773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.478843 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.479449 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:49.479457 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:49.481073 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:49.482892 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:49.482901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:49.482903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:49.483123 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:49.483402 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:49.483403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:49.483403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:49.483475 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:49.483806 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:49.483814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:49.483913 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.483989 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.484133 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":137,"threads_started":1}
I20260807 15:01:49.484282 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.476048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.1.127.254:39097)
0807 15:01:49.476137 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.476140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.476489 (+ 349us) server_negotiation.cc:407] Connection header received
0807 15:01:49.476565 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.476566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.476576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.476653 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:49.477147 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.477657 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.478386 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.478576 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.478642 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.481170 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:49.481171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:49.481171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:49.481178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:49.482737 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.483214 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.483215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.483215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.483276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.483571 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.483572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.483572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.483717 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:49.483784 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.484088 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.484188 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":57}
I20260807 15:01:49.487936 2712 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:49632:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:01:49.488257 2712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:49.490042 2857 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdfc0b0b0d4aab803d8fbb1303b312 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75572f98be642e38ad841a130030e30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:49.490087 3003 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdfc0b0b0d4aab803d8fbb1303b312 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75572f98be642e38ad841a130030e30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:49.490078 3146 tablet_service.cc:1467] Processing CreateTablet for tablet 35cdfc0b0b0d4aab803d8fbb1303b312 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75572f98be642e38ad841a130030e30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:49.490278 3146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdfc0b0b0d4aab803d8fbb1303b312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.490278 3003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdfc0b0b0d4aab803d8fbb1303b312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.490306 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cdfc0b0b0d4aab803d8fbb1303b312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:49.491897 3241 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: Bootstrap starting.
I20260807 15:01:49.492008 3242 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: Bootstrap starting.
I20260807 15:01:49.492012 3245 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: Bootstrap starting.
I20260807 15:01:49.492393 3245 tablet_bootstrap.cc:654] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.492434 3241 tablet_bootstrap.cc:654] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.492434 3242 tablet_bootstrap.cc:654] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:49.493126 3245 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: No bootstrap required, opened a new log
I20260807 15:01:49.493155 3241 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: No bootstrap required, opened a new log
I20260807 15:01:49.493202 3241 ts_tablet_manager.cc:1397] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260807 15:01:49.493201 3245 ts_tablet_manager.cc:1397] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:49.493343 3245 raft_consensus.cc:348] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.493362 3241 raft_consensus.cc:348] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.493429 3241 raft_consensus.cc:374] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.493432 3245 raft_consensus.cc:374] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.493444 3241 raft_consensus.cc:729] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a22b367bc247d6bf940b99f40045ef, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.493460 3245 raft_consensus.cc:729] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5516ac65ffca43e0ae322adc65c9b53b, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.493520 3241 consensus_queue.cc:260] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.493525 3245 consensus_queue.cc:260] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.493618 3245 ts_tablet_manager.cc:1428] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:01:49.493631 3241 ts_tablet_manager.cc:1428] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:49.493710 3247 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.493738 3249 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.493799 3247 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.493825 3249 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.494390 3242 tablet_bootstrap.cc:492] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: No bootstrap required, opened a new log
I20260807 15:01:49.494457 3242 ts_tablet_manager.cc:1397] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 15:01:49.494756 3242 raft_consensus.cc:348] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.494849 3242 raft_consensus.cc:374] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:49.494865 3242 raft_consensus.cc:729] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e0eba8ffeb4cfab62744eef6f29a65, State: Initialized, Role: FOLLOWER
I20260807 15:01:49.494937 3242 consensus_queue.cc:260] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.495035 3242 ts_tablet_manager.cc:1428] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:01:49.495097 3248 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.495213 3248 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.534173 3249 raft_consensus.cc:482] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:49.534281 3249 raft_consensus.cc:504] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.534437 3249 leader_election.cc:288] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919)
I20260807 15:01:49.534757 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cdfc0b0b0d4aab803d8fbb1303b312" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef" is_pre_election: true
I20260807 15:01:49.534866 2869 raft_consensus.cc:2454] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 0.
I20260807 15:01:49.534834 3167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cdfc0b0b0d4aab803d8fbb1303b312" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" is_pre_election: true
I20260807 15:01:49.534970 3167 raft_consensus.cc:2454] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 0.
I20260807 15:01:49.535132 2952 leader_election.cc:302] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5516ac65ffca43e0ae322adc65c9b53b, b2a22b367bc247d6bf940b99f40045ef; no voters:
I20260807 15:01:49.535236 3249 raft_consensus.cc:2781] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:49.535280 3249 raft_consensus.cc:482] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:49.535313 3249 raft_consensus.cc:3037] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.536000 3249 raft_consensus.cc:504] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.536140 3249 leader_election.cc:288] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 election: Requested vote from peers b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333), 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919)
I20260807 15:01:49.536304 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cdfc0b0b0d4aab803d8fbb1303b312" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a22b367bc247d6bf940b99f40045ef"
I20260807 15:01:49.536401 2869 raft_consensus.cc:3037] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.536370 3167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cdfc0b0b0d4aab803d8fbb1303b312" candidate_uuid: "5516ac65ffca43e0ae322adc65c9b53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e0eba8ffeb4cfab62744eef6f29a65"
I20260807 15:01:49.536451 3167 raft_consensus.cc:3037] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:49.537210 3167 raft_consensus.cc:2454] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 1.
I20260807 15:01:49.537210 2869 raft_consensus.cc:2454] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5516ac65ffca43e0ae322adc65c9b53b in term 1.
I20260807 15:01:49.537405 2952 leader_election.cc:302] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5516ac65ffca43e0ae322adc65c9b53b, 77e0eba8ffeb4cfab62744eef6f29a65; no voters:
I20260807 15:01:49.537556 3249 raft_consensus.cc:2781] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:49.537618 3249 raft_consensus.cc:686] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [term 1 LEADER]: Becoming Leader. State: Replica: 5516ac65ffca43e0ae322adc65c9b53b, State: Running, Role: LEADER
I20260807 15:01:49.537678 3249 consensus_queue.cc:237] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } }
I20260807 15:01:49.538055 3253 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: TxnStatusTablet state changed. Reason: New leader 5516ac65ffca43e0ae322adc65c9b53b. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.538163 3253 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:49.538343 3286 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:01:49.538378 2712 catalog_manager.cc:5495] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194). New cstate: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260807 15:01:49.538532 3286 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:01:49.556347 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.553356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39377 (local address 127.0.0.1:39146)
0807 15:01:49.553478 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.553524 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.553599 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.553777 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.553778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.553779 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:49.553835 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.553837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.554793 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.554795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.555447 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.555450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.555939 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.555940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.555984 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.556004 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.556187 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.556195 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.556237 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.556278 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 15:01:49.556358 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.553486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.1.127.194:39377)
0807 15:01:49.553571 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.553573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.553585 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:01:49.553681 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.553682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.553694 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.553758 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:49.553903 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.554700 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.555548 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.555865 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.555931 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.556088 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.556123 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.556176 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.556240 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.556272 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:01:49.557709 3167 raft_consensus.cc:1261] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Refusing update from remote peer 5516ac65ffca43e0ae322adc65c9b53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.558053 2869 raft_consensus.cc:1261] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Refusing update from remote peer 5516ac65ffca43e0ae322adc65c9b53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.558075 3253 consensus_queue.cc:1035] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.558341 3253 consensus_queue.cc:1035] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.558725 3248 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: TxnStatusTablet state changed. Reason: New leader 5516ac65ffca43e0ae322adc65c9b53b. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.558823 3248 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.559346 3247 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: TxnStatusTablet state changed. Reason: New leader 5516ac65ffca43e0ae322adc65c9b53b. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.559448 3247 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.559769 3249 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.559875 3249 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:49.560117 3256 mvcc.cc:205] Tried to move back new op lower bound from 7315926669546315776 to 7315926669466841088. Current Snapshot: MvccSnapshot[applied={T|T < 7315926669546315776 or (T in {7315926669546315776})}]
I20260807 15:01:49.560491 3253 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.560566 3253 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 5516ac65ffca43e0ae322adc65c9b53b: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:49.560894 3247 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.560977 3247 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P b2a22b367bc247d6bf940b99f40045ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.561349 3248 tablet_replica.cc:439] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5516ac65ffca43e0ae322adc65c9b53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 } } peers { permanent_uuid: "77e0eba8ffeb4cfab62744eef6f29a65" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41919 } } peers { permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 } } }
I20260807 15:01:49.561549 3248 tablet_replica.cc:442] T 35cdfc0b0b0d4aab803d8fbb1303b312 P 77e0eba8ffeb4cfab62744eef6f29a65: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:49.566493 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.563675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.1.127.195:41919)
0807 15:01:49.563716 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.563718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.563728 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:49.563820 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.563821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.563846 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.563938 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:49.564081 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.564809 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.565588 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.565892 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.565966 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.566135 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.566269 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.566347 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.566371 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.566393 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:01:49.572160 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.569379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:39993 (local address 127.1.127.194:39377)
0807 15:01:49.569453 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.569455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.569726 (+ 271us) server_negotiation.cc:407] Connection header received
0807 15:01:49.569815 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.569816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.569828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.569969 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:49.570124 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.570600 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.571378 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.571583 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.571672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.571878 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.571919 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.571974 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.572049 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.572073 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:01:49.572216 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.569322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39377 (local address 127.1.127.195:39993)
0807 15:01:49.569510 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.569519 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.569741 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.569977 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.569977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.569979 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:49.570047 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.570048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.570717 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.570718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.571289 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.571291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.571675 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.571676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.571754 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.571770 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.571988 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.571990 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.572037 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.572085 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260807 15:01:49.575497 3024 raft_consensus.cc:1261] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Refusing update from remote peer 77e0eba8ffeb4cfab62744eef6f29a65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.575790 2869 raft_consensus.cc:1261] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Refusing update from remote peer 77e0eba8ffeb4cfab62744eef6f29a65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:49.575830 3272 consensus_queue.cc:1035] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5516ac65ffca43e0ae322adc65c9b53b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.575997 3248 consensus_queue.cc:1035] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a22b367bc247d6bf940b99f40045ef" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:49.577577 3259 mvcc.cc:205] Tried to move back new op lower bound from 7315926669618798592 to 7315926668990251008. Current Snapshot: MvccSnapshot[applied={T|T < 7315926669618798592}]
I20260807 15:01:49.587646 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.584131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41919 (local address 127.1.127.194:33599)
0807 15:01:49.584369 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:49.584383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:49.584568 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:49.584738 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:49.584740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:49.584741 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:49.584794 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.584796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.585706 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.585707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.586639 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:49.586642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.587176 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.587177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:49.587234 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.587252 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.587458 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.587459 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:01:49.587505 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:01:49.587555 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":139,"threads_started":1}
I20260807 15:01:49.587672 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.584375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:33599 (local address 127.1.127.195:41919)
0807 15:01:49.584414 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.584416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.584571 (+ 155us) server_negotiation.cc:407] Connection header received
0807 15:01:49.584623 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.584624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.584634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.584734 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:49.584860 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.585615 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.586780 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.587074 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.587116 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.587342 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:49.587387 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:49.587416 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.587586 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.587609 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:01:49.596637 2712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:49.596768 2712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:49.597419 2712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53c7fd63c2fc42e889aa6bad378d9aed: Sending DeleteTablet for 3 replicas of tablet 4ddd224fe16c4bf5986d267470ec59ee
I20260807 15:01:49.597873 2857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260807 15:01:49.598032 3146 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:49306
I20260807 15:01:49.598140 3003 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ddd224fe16c4bf5986d267470ec59ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:49 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39128
I20260807 15:01:49.598137 3267 tablet_replica.cc:330] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: stopping tablet replica
I20260807 15:01:49.598210 3267 raft_consensus.cc:2229] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:49.598253 3265 tablet_replica.cc:330] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: stopping tablet replica
I20260807 15:01:49.598291 3267 raft_consensus.cc:2258] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.598331 3265 raft_consensus.cc:2229] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:49.598373 3265 raft_consensus.cc:2258] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.598443 3266 tablet_replica.cc:330] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: stopping tablet replica
I20260807 15:01:49.598506 3266 raft_consensus.cc:2229] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:49.598547 3266 raft_consensus.cc:2258] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:49.599098 3266 ts_tablet_manager.cc:1905] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.600706 3265 ts_tablet_manager.cc:1905] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.602136 3266 ts_tablet_manager.cc:1918] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:49.602205 3266 log.cc:1196] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/4ddd224fe16c4bf5986d267470ec59ee
I20260807 15:01:49.602254 3265 ts_tablet_manager.cc:1918] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:49.602319 3265 log.cc:1196] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/4ddd224fe16c4bf5986d267470ec59ee
I20260807 15:01:49.602411 3267 ts_tablet_manager.cc:1905] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:49.602509 3266 ts_tablet_manager.cc:1939] T 4ddd224fe16c4bf5986d267470ec59ee P b2a22b367bc247d6bf940b99f40045ef: Deleting consensus metadata
I20260807 15:01:49.602563 3265 ts_tablet_manager.cc:1939] T 4ddd224fe16c4bf5986d267470ec59ee P 5516ac65ffca43e0ae322adc65c9b53b: Deleting consensus metadata
I20260807 15:01:49.602974 2695 catalog_manager.cc:4841] TS b2a22b367bc247d6bf940b99f40045ef (127.1.127.193:40333): tablet 4ddd224fe16c4bf5986d267470ec59ee (table test-table [id=14cd5f490d3d443580861f4a9dbfc750]) successfully deleted
I20260807 15:01:49.603133 2697 catalog_manager.cc:4841] TS 5516ac65ffca43e0ae322adc65c9b53b (127.1.127.194:39377): tablet 4ddd224fe16c4bf5986d267470ec59ee (table test-table [id=14cd5f490d3d443580861f4a9dbfc750]) successfully deleted
I20260807 15:01:49.604511 3267 ts_tablet_manager.cc:1918] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:49.604588 3267 log.cc:1196] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/4ddd224fe16c4bf5986d267470ec59ee
I20260807 15:01:49.604858 3267 ts_tablet_manager.cc:1939] T 4ddd224fe16c4bf5986d267470ec59ee P 77e0eba8ffeb4cfab62744eef6f29a65: Deleting consensus metadata
I20260807 15:01:49.605234 2695 catalog_manager.cc:4841] TS 77e0eba8ffeb4cfab62744eef6f29a65 (127.1.127.195:41919): tablet 4ddd224fe16c4bf5986d267470ec59ee (table test-table [id=14cd5f490d3d443580861f4a9dbfc750]) successfully deleted
Aug 07 15:01:49 dist-test-slave-8tdl krb5kdc[2665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:01:49.616050 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.604451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.1.127.193:40333)
0807 15:01:49.604503 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.604505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.607009 (+ 2504us) server_negotiation.cc:407] Connection header received
0807 15:01:49.607205 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.607206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.607221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.607290 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:49.607922 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.608424 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.609299 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.609653 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.609704 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.612881 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:49.612903 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:49.612903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:49.612930 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:49.614776 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.615179 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.615184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.615184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.615248 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.615468 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.615469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.615469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.615632 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:49.615793 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.615836 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.615932 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:01:49.616408 2913 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36708
I20260807 15:01:49.625574 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:49.618993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.1.127.254:39097)
0807 15:01:49.619044 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:49.619046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:49.619170 (+ 124us) server_negotiation.cc:407] Connection header received
0807 15:01:49.619251 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:49.619252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:49.619264 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:49.619335 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:49.619879 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.620389 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.621162 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:49.621368 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:49.621459 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:49.622779 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:49.622780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:49.622781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:49.622789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:49.624333 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.624748 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.624749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.624750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.624817 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:49.625034 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:49.625035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:49.625036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:49.625156 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:49.625266 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:49.625364 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 15:01:49.625458 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:01:49.625818 2712 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49638
I20260807 15:01:49.626515 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2789
I20260807 15:01:49.640458 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2935
I20260807 15:01:49.649115 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 3081
I20260807 15:01:49.655543 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 2678
2026-08-07T15:01:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2662 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:51 dist-test-slave-8tdl krb5kdc[3310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:51 dist-test-slave-8tdl krb5kdc[3310](info): set up 2 sockets
Aug 07 15:01:51 dist-test-slave-8tdl krb5kdc[3310](info): commencing operation
krb5kdc: starting...
W20260807 15:01:52.207083 1535 mini_kdc.cc:121] Time spent starting KDC: real 2.528s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:52.347246 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33227
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:33227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:52.439199 3323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:52.439479 3323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:52.439538 3323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:52.440752 3323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:01:52.440843 3323 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:01:52.440867 3323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:52.440891 3323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:52.440922 3323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:01:52.440945 3323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:01:52.442598 3323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:33227
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:52.442934 3323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:52.443246 3323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:52.448733 3328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.448745 3331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.448745 3329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:52.448873 3323 server_base.cc:1034] running on GCE node
I20260807 15:01:52.449419 3323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:52.449740 3323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:52.450914 3323 hybrid_clock.cc:648] HybridClock initialized: now 1786114912450892 us; error 41 us; skew 500 ppm
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:52.453781 3323 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:01:52.454289 3323 webserver.cc:460] Webserver started at http://127.1.127.254:36835/ using document root <none> and password file <none>
I20260807 15:01:52.454552 3323 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:52.454602 3323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:52.454782 3323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:52.455976 3323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "3c212a87052c442481a7d40dfcbd9d0e"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "118ae9fcfc1efabc77630591980b8c4c"
server_key_iv: "faf52ff37b365c9c75d550c91f66a703"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.456377 3323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "3c212a87052c442481a7d40dfcbd9d0e"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "118ae9fcfc1efabc77630591980b8c4c"
server_key_iv: "faf52ff37b365c9c75d550c91f66a703"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.457906 3323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:52.458885 3338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:52.459125 3323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:52.459239 3323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "3c212a87052c442481a7d40dfcbd9d0e"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "118ae9fcfc1efabc77630591980b8c4c"
server_key_iv: "faf52ff37b365c9c75d550c91f66a703"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.459331 3323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:52.471148 3323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:52.473786 3323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:52.473973 3323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:52.478996 3323 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:33227
I20260807 15:01:52.479068 3406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:33227 every 8 connection(s)
I20260807 15:01:52.479436 3323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:01:52.480247 3407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:52.482797 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Bootstrap starting.
I20260807 15:01:52.483448 3407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:52.483784 3407 log.cc:824] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:52.483960 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 3323
I20260807 15:01:52.484102 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:01:52.484299 1535 external_mini_cluster.cc:1428] Setting key 3ba0c3d6d634d0965d492fbbb221a666
I20260807 15:01:52.484609 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: No bootstrap required, opened a new log
I20260807 15:01:52.487058 3407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } }
I20260807 15:01:52.487238 3407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:52.487308 3407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c212a87052c442481a7d40dfcbd9d0e, State: Initialized, Role: FOLLOWER
I20260807 15:01:52.487485 3407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } }
I20260807 15:01:52.487587 3407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:01:52.487634 3407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:01:52.487690 3407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:52.488277 3407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } }
I20260807 15:01:52.488399 3407 leader_election.cc:302] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c212a87052c442481a7d40dfcbd9d0e; no voters:
I20260807 15:01:52.488644 3407 leader_election.cc:288] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:01:52.488678 3412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:52.488804 3412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [term 1 LEADER]: Becoming Leader. State: Replica: 3c212a87052c442481a7d40dfcbd9d0e, State: Running, Role: LEADER
I20260807 15:01:52.488955 3407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:01:52.488962 3412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } }
I20260807 15:01:52.489516 3413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c212a87052c442481a7d40dfcbd9d0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } } }
I20260807 15:01:52.489612 3413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:52.489849 3414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c212a87052c442481a7d40dfcbd9d0e. Latest consensus state: current_term: 1 leader_uuid: "3c212a87052c442481a7d40dfcbd9d0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c212a87052c442481a7d40dfcbd9d0e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33227 } } }
I20260807 15:01:52.489919 3414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e [sys.catalog]: This master's current role is: LEADER
W20260807 15:01:52.491304 3427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:01:52.491987 3427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:01:52.492131 3421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:01:52.492419 3421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:52.494144 3421 catalog_manager.cc:1347] Generated new cluster ID: cb71794465c84c6bb4f37915da765f1d
I20260807 15:01:52.494200 3421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:01:52.498342 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.484892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.1.127.254:33227)
0807 15:01:52.485110 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:52.485123 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:52.485151 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:01:52.487050 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:52.487059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:52.487122 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:52.487319 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:52.488104 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.489364 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.490380 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.491312 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.491419 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.494396 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:52.494399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:52.494400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:52.494411 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:52.496499 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.496975 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.496978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.496979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.497127 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.497366 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.497366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.497367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.497675 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:52.497822 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:52.497980 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 15:01:52.498129 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260807 15:01:52.515985 3421 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:01:52.516781 3421 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:01:52.520685 3421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Generated new TSK 0
I20260807 15:01:52.520917 3421 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:01:52.558817 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33227
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:52.663892 3433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:52.664172 3433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:52.664220 3433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:52.665551 3433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:52.665617 3433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:52.665650 3433 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:01:52.665696 3433 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:52.665735 3433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:01:52.667403 3433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:52.667702 3433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:52.667961 3433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:52.673606 3438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.673606 3441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.673610 3439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:52.673887 3433 server_base.cc:1034] running on GCE node
I20260807 15:01:52.674211 3433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:52.674602 3433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:52.675817 3433 hybrid_clock.cc:648] HybridClock initialized: now 1786114912675802 us; error 73 us; skew 500 ppm
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:52.679030 3433 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:01:52.679561 3433 webserver.cc:460] Webserver started at http://127.1.127.193:41849/ using document root <none> and password file <none>
I20260807 15:01:52.679812 3433 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:52.679889 3433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:52.680051 3433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:52.681293 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "222a77be2f6d470db42bfad8e6cb7174"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "150a092061fe425fbbf37afd0a724ff9"
server_key_iv: "3ccfef9485d3abcc393ca1e08937219c"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.681699 3433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "222a77be2f6d470db42bfad8e6cb7174"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "150a092061fe425fbbf37afd0a724ff9"
server_key_iv: "3ccfef9485d3abcc393ca1e08937219c"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.683327 3433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:52.684257 3448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:52.684486 3433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:52.684605 3433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "222a77be2f6d470db42bfad8e6cb7174"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "150a092061fe425fbbf37afd0a724ff9"
server_key_iv: "3ccfef9485d3abcc393ca1e08937219c"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.684689 3433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:52.721264 3433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:52.724148 3433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:52.724372 3433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:52.725081 3433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:52.725158 3433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:52.725196 3433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:52.725232 3433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:52.734565 3433 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:32865
I20260807 15:01:52.734658 3570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:32865 every 8 connection(s)
I20260807 15:01:52.735046 3433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:01:52.736593 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 3433
I20260807 15:01:52.736673 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:01:52.736884 1535 external_mini_cluster.cc:1428] Setting key 3f20230a4bd4687591d950d7205865d3
I20260807 15:01:52.738256 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.725659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:49247 (local address 127.1.127.254:33227)
0807 15:01:52.725741 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:52.725744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:52.726070 (+ 326us) server_negotiation.cc:407] Connection header received
0807 15:01:52.726710 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:52.726712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:52.726732 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:52.726831 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:52.727531 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.728056 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.729821 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.730178 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.730254 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.734299 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:52.734301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:52.734301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:52.734372 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:52.736364 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.736955 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.736957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.736958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.737034 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.737361 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.737362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.737362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.737484 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:52.737625 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:52.738036 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 15:01:52.738145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:01:52.738498 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.725633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.193:49247)
0807 15:01:52.725825 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:52.725868 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:52.726565 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:52.726910 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:52.726913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:52.727216 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:52.727424 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.727428 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.728163 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.728165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.729722 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.729726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.730294 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.730295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.730509 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.731315 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:52.731329 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:52.734120 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:52.736482 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.736491 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.736494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.736837 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.737122 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.737123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.737123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.737289 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.737720 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:52.737727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:52.737867 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 15:01:52.737997 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 15:01:52.738214 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260807 15:01:52.739035 3571 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33227
I20260807 15:01:52.739166 3571 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:52.739429 3571 heartbeater.cc:511] Master 127.1.127.254:33227 requested a full tablet report, sending...
I20260807 15:01:52.740339 3357 ts_manager.cc:194] Registered new tserver with Master: 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865)
I20260807 15:01:52.741130 3357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:49247
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:01:52.747778 3578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.740264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.193:40943)
0807 15:01:52.740463 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:52.740478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:52.740538 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:52.741500 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:52.741501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:52.741690 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:52.741758 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.741760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.742531 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.742532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.743302 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.743307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.743767 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.743768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.743875 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.744446 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:52.744451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:52.744796 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:52.746649 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.746650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.746650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.746877 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.747109 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.747110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.747110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.747187 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.747453 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:52.747454 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:52.747508 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:01:52.747583 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:01:52.747693 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260807 15:01:52.748780 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.740449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:40943 (local address 127.1.127.254:33227)
0807 15:01:52.741348 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:52.741349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:52.741358 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:01:52.741377 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:52.741377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:52.741386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:52.741483 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:52.741845 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.742358 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.743438 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.743664 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.743765 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.744941 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:52.744943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:52.744943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:52.744952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:52.746530 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.746965 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.746966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.746967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.747031 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.747257 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.747258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.747258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.747381 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:52.747464 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:52.747642 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 15:01:52.747747 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:01:52.770995 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33227
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:52.856556 3580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:52.856825 3580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:52.856868 3580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:52.858098 3580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:52.858175 3580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:52.858212 3580 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:01:52.858239 3580 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:52.858283 3580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:01:52.859938 3580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:52.860253 3580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:52.860522 3580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:52.866274 3585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.866343 3588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:52.866266 3586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:52.866587 3580 server_base.cc:1034] running on GCE node
I20260807 15:01:52.866835 3580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:52.867136 3580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:52.868371 3580 hybrid_clock.cc:648] HybridClock initialized: now 1786114912868324 us; error 39 us; skew 500 ppm
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:52.871475 3580 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:01:52.871971 3580 webserver.cc:460] Webserver started at http://127.1.127.194:36575/ using document root <none> and password file <none>
I20260807 15:01:52.872254 3580 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:52.872341 3580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:52.872503 3580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:52.873667 3580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "5745462d6b01477f8418f5f872afbefd"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "a2ca4fe12e8148100148a823513d3ca8"
server_key_iv: "f37aa12a96c94aa9cf42671388ddce89"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.874051 3580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "5745462d6b01477f8418f5f872afbefd"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "a2ca4fe12e8148100148a823513d3ca8"
server_key_iv: "f37aa12a96c94aa9cf42671388ddce89"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.875677 3580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:01:52.876626 3595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:52.876993 3580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:52.877123 3580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "5745462d6b01477f8418f5f872afbefd"
format_stamp: "Formatted at 2026-08-07 15:01:52 on dist-test-slave-8tdl"
server_key: "a2ca4fe12e8148100148a823513d3ca8"
server_key_iv: "f37aa12a96c94aa9cf42671388ddce89"
server_key_version: "encryptionkey@0"
I20260807 15:01:52.877214 3580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:52.907003 3580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:52.909379 3580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:52.909559 3580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:52.910136 3580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:52.910205 3580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:52.910255 3580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:52.910310 3580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:01:52 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:52.918467 3580 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:36535
I20260807 15:01:52.918606 3719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:36535 every 8 connection(s)
I20260807 15:01:52.919026 3580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:01:52.921653 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.910886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:44987 (local address 127.1.127.254:33227)
0807 15:01:52.910966 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:52.910969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:52.911245 (+ 276us) server_negotiation.cc:407] Connection header received
0807 15:01:52.912067 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:52.912069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:52.912084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:52.912158 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:52.912974 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.913844 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.915105 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.915312 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.915405 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.917894 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:52.917896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:52.917896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:52.917905 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:52.919772 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.920327 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.920328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.920329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.920404 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.920772 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.920773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.920774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.920920 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:52.921068 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:52.921483 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 15:01:52.921583 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:01:52.921788 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.910864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.194:44987)
0807 15:01:52.911082 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:52.911135 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:52.911919 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:52.912257 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:52.912261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:52.912534 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:52.912840 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.912846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.913994 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.913996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.915002 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.915005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.915439 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.915440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.915575 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.916176 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:52.916185 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:52.917701 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:52.919887 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.919894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.919897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.920214 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.920534 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.920535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.920536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.920672 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.921109 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:52.921117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:52.921274 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 15:01:52.921412 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 15:01:52.921566 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":110,"threads_started":1}
I20260807 15:01:52.922362 3721 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33227
I20260807 15:01:52.922451 3721 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:52.922653 3721 heartbeater.cc:511] Master 127.1.127.254:33227 requested a full tablet report, sending...
I20260807 15:01:52.923142 3358 ts_manager.cc:194] Registered new tserver with Master: 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:52.923563 3358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:44987
I20260807 15:01:52.927525 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 3580
I20260807 15:01:52.927611 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:01:52.927822 1535 external_mini_cluster.cc:1428] Setting key 88e065cb04ab623a2b6282097b171682
I20260807 15:01:52.930318 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.923609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.194:48439)
0807 15:01:52.923754 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:52.923767 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:52.923829 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:52.923970 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:52.923971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:52.924205 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:52.924276 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.924279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.924958 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.924959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.925500 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:52.925502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.925830 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.925831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:52.925904 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.926428 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:52.926433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:52.926842 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:52.929121 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.929121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.929122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.929313 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.929592 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:52.929593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:52.929593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:52.929661 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:52.930037 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:52.930038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:52.930084 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:01:52.930147 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:01:52.930242 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260807 15:01:52.930538 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:52.923662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:48439 (local address 127.1.127.254:33227)
0807 15:01:52.923707 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:52.923708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:52.923830 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:01:52.923887 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:52.923888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:52.923900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:52.923956 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:52.924356 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.924844 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.925578 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:52.925756 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:52.925806 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:52.926926 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:52.926927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:52.926927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:52.926934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:52.929028 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.929394 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.929395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.929395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.929496 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:52.929737 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:52.929738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:52.929739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:52.929960 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:52.930034 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:52.930192 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 15:01:52.930283 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:01:52.959511 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33227
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:53.048434 3730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:53.048669 3730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:53.048697 3730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:53.049850 3730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:53.049902 3730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:53.049924 3730 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:01:53.049937 3730 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:01:53.049954 3730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:01:53.051616 3730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:53.051860 3730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:53.052059 3730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:01:53.057420 3730 server_base.cc:1034] running on GCE node
W20260807 15:01:53.057436 3735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:53.057564 3738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:53.057440 3736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:53.058022 3730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:53.058290 3730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:53.059485 3730 hybrid_clock.cc:648] HybridClock initialized: now 1786114913059460 us; error 58 us; skew 500 ppm
Aug 07 15:01:53 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:53.062559 3730 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:01:53.063150 3730 webserver.cc:460] Webserver started at http://127.1.127.195:34731/ using document root <none> and password file <none>
I20260807 15:01:53.063421 3730 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:53.063496 3730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:53.063680 3730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:53.064903 3730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "81ef28d4b83a428abdc1f2265581b6a0"
format_stamp: "Formatted at 2026-08-07 15:01:53 on dist-test-slave-8tdl"
server_key: "937f4d7aeeb1bc4d7c4b09cbc42e68c8"
server_key_iv: "dee32a57def45bad0b73a3c27a042563"
server_key_version: "encryptionkey@0"
I20260807 15:01:53.065332 3730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "81ef28d4b83a428abdc1f2265581b6a0"
format_stamp: "Formatted at 2026-08-07 15:01:53 on dist-test-slave-8tdl"
server_key: "937f4d7aeeb1bc4d7c4b09cbc42e68c8"
server_key_iv: "dee32a57def45bad0b73a3c27a042563"
server_key_version: "encryptionkey@0"
I20260807 15:01:53.066941 3730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:01:53.067709 3745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.067898 3730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:53.068032 3730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "81ef28d4b83a428abdc1f2265581b6a0"
format_stamp: "Formatted at 2026-08-07 15:01:53 on dist-test-slave-8tdl"
server_key: "937f4d7aeeb1bc4d7c4b09cbc42e68c8"
server_key_iv: "dee32a57def45bad0b73a3c27a042563"
server_key_version: "encryptionkey@0"
I20260807 15:01:53.068096 3730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:53.084430 3730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:53.087159 3730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:53.087338 3730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:53.088020 3730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:53.088078 3730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.088102 3730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:53.088114 3730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.095501 3730 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:43455
I20260807 15:01:53.095645 3866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:43455 every 8 connection(s)
I20260807 15:01:53.095935 3730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:01:53 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114913, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:53.100157 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.088596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:37417 (local address 127.1.127.254:33227)
0807 15:01:53.088667 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.088669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.088981 (+ 312us) server_negotiation.cc:407] Connection header received
0807 15:01:53.089674 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.089675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.089691 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.089759 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:53.090877 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.091821 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.093522 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.093732 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.093798 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.096726 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:53.096727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:53.096727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:53.096735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:53.098454 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.099095 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.099096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.099097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.099162 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.099456 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.099457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.099457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.099583 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:53.099638 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.100001 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.100088 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:01:53.100368 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.088590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.195:37417)
0807 15:01:53.088766 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.088806 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.089543 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.089866 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.089870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.090356 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:53.090717 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.090726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.091940 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.091941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.093420 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.093423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.093828 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.093829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.093995 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.094665 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:53.094706 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:53.096608 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:53.098566 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.098574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.098576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.098992 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.099260 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.099261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.099262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.099382 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.099665 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:53.099672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:53.099820 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.099916 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.100106 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260807 15:01:53.101075 3867 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33227
I20260807 15:01:53.101163 3867 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:53.101372 3867 heartbeater.cc:511] Master 127.1.127.254:33227 requested a full tablet report, sending...
I20260807 15:01:53.101759 3358 ts_manager.cc:194] Registered new tserver with Master: 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455)
I20260807 15:01:53.102310 3358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:37417
I20260807 15:01:53.105993 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 3730
I20260807 15:01:53.106093 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:01:53.106307 1535 external_mini_cluster.cc:1428] Setting key b9556750c49b9667566123e1ee0442e2
I20260807 15:01:53.107304 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:01:53.108630 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.102275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:51481 (local address 127.1.127.254:33227)
0807 15:01:53.102319 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.102321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.102387 (+ 66us) server_negotiation.cc:407] Connection header received
0807 15:01:53.102512 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.102513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.102523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.102581 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:53.102951 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.103433 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.104143 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.104347 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.104424 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.105426 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:53.105428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:53.105428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:53.105435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:53.107199 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.107631 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.107633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.107633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.107708 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.107975 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.107976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.107976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.108141 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:53.108222 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.108414 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.108520 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:01:53.108626 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.102176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.1.127.195:51481)
0807 15:01:53.102322 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.102335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.102404 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.102585 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.102586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.102812 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:53.102867 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.102869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.103536 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.103537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.104046 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.104048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.104423 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.104424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.104513 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.105004 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:53.105009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:53.105314 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:53.107293 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.107294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.107294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.107555 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.107792 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.107792 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.107793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.107897 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.108225 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:53.108227 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:53.108298 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.108383 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.108515 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
Aug 07 15:01:53 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:01:53 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:53.133246 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.125408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.1.127.254:33227)
0807 15:01:53.125471 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.125474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.125515 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:01:53.125542 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.125543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.125552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.125597 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:53.126402 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.127027 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.127830 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.128030 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.128095 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.130604 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:53.130605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:53.130605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:53.130613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:53.132183 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.132542 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.132543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.132543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.132601 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.132828 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.132829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.132829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.132935 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:53.133016 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.133113 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.133177 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:01:53.135342 3358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:53.137247 3358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:53.144338 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.1.127.193:32865)
0807 15:01:53.140578 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.140582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.140590 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:01:53.140627 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.140630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.140645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.140938 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.141007 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.141722 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.143251 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.143896 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144009 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144080 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.144193 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.144234 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":94,"threads_started":1}
I20260807 15:01:53.144332 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:32865 (local address 127.0.0.1:46646)
0807 15:01:53.140206 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.140253 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.140330 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.140748 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.140749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.140752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.140823 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.140825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.141822 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.141823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.143157 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.143160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.144002 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.144088 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144103 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.144179 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.144234 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":11,"thread_start_us":73,"threads_started":1}
I20260807 15:01:53.144605 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.0.0.1:39110)
0807 15:01:53.140406 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.140416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.140472 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.140724 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.140727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.140730 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.140811 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.140815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.142211 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.142212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.143285 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.143287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.144320 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.144400 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144404 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.144468 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.144517 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260807 15:01:53.144800 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.1.127.194:36535)
0807 15:01:53.140611 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.140615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.140969 (+ 354us) server_negotiation.cc:407] Connection header received
0807 15:01:53.141032 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.141035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.141051 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.141121 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.141304 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.142310 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.143542 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.144251 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144314 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144363 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.144653 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.144699 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
I20260807 15:01:53.144846 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.1.127.195:43455)
0807 15:01:53.140524 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.140529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.140541 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:01:53.140585 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.140587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.140605 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.140694 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.140916 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.142083 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.143386 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.144148 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144224 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144280 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.144702 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.144744 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260807 15:01:53.144605 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.140279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:36535 (local address 127.0.0.1:45098)
0807 15:01:53.140651 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.140859 (+ 208us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.140906 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.141189 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.141190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.141192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.141231 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.141232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.142389 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.142391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.143441 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.143444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.144395 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.144396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.144452 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.144456 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.144503 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.144532 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":74,"threads_started":1}
I20260807 15:01:53.145608 3796 tablet_service.cc:1467] Processing CreateTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 (DEFAULT_TABLE table=test-table [id=ea15216e107c4265b1cc4ef8b7756a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.145864 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 (DEFAULT_TABLE table=test-table [id=ea15216e107c4265b1cc4ef8b7756a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.145867 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 (DEFAULT_TABLE table=test-table [id=ea15216e107c4265b1cc4ef8b7756a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.146003 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513ffdb3f47142fdb5edd045d23d7998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.146250 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513ffdb3f47142fdb5edd045d23d7998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.146250 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513ffdb3f47142fdb5edd045d23d7998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.148874 3890 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Bootstrap starting.
I20260807 15:01:53.149956 3890 tablet_bootstrap.cc:654] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.150245 3892 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Bootstrap starting.
I20260807 15:01:53.150499 3890 log.cc:824] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:53.150837 3891 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Bootstrap starting.
I20260807 15:01:53.150964 3892 tablet_bootstrap.cc:654] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.151268 3892 log.cc:824] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:53.151381 3890 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: No bootstrap required, opened a new log
I20260807 15:01:53.151464 3890 ts_tablet_manager.cc:1397] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 15:01:53.151566 3891 tablet_bootstrap.cc:654] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.151916 3891 log.cc:824] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:53.152082 3892 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: No bootstrap required, opened a new log
I20260807 15:01:53.152173 3892 ts_tablet_manager.cc:1397] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:53.152778 3891 tablet_bootstrap.cc:492] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: No bootstrap required, opened a new log
I20260807 15:01:53.152902 3891 ts_tablet_manager.cc:1397] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:01:53.153873 3892 raft_consensus.cc:348] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.153898 3890 raft_consensus.cc:348] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.154048 3892 raft_consensus.cc:374] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.154060 3890 raft_consensus.cc:374] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.154098 3892 raft_consensus.cc:729] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.154096 3890 raft_consensus.cc:729] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.154243 3892 consensus_queue.cc:260] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.154245 3890 consensus_queue.cc:260] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.154575 3571 heartbeater.cc:503] Master 127.1.127.254:33227 was elected leader, sending a full tablet report...
I20260807 15:01:53.154625 3890 ts_tablet_manager.cc:1428] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:01:53.154894 3867 heartbeater.cc:503] Master 127.1.127.254:33227 was elected leader, sending a full tablet report...
I20260807 15:01:53.155179 3892 ts_tablet_manager.cc:1428] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:01:53.155215 3891 raft_consensus.cc:348] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.155370 3891 raft_consensus.cc:374] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.155462 3891 raft_consensus.cc:729] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.155599 3891 consensus_queue.cc:260] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.155877 3891 ts_tablet_manager.cc:1428] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:01:53.155952 3721 heartbeater.cc:503] Master 127.1.127.254:33227 was elected leader, sending a full tablet report...
W20260807 15:01:53.169757 3722 tablet.cc:2369] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:53.173628 3896 raft_consensus.cc:482] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:53.173759 3896 raft_consensus.cc:504] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.174134 3896 leader_election.cc:288] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:53.177395 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.174406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.1.127.193:58939)
0807 15:01:53.174469 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.174483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.174561 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.174912 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.174914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.174916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.175005 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.175007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.175689 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.175691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.176536 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.176539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.177096 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.177096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.177189 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.177192 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.177262 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.177310 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":57}
I20260807 15:01:53.177582 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.174445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:58939 (local address 127.1.127.195:43455)
0807 15:01:53.174496 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.174498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.174544 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:01:53.174665 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.174666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.174723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.174776 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.175100 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.175592 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.176644 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.177009 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.177097 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.177106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.177434 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.177503 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:01:53.177762 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.174553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:55239 (local address 127.1.127.194:36535)
0807 15:01:53.174610 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.174612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.174847 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:01:53.174928 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.174929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.174937 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.174977 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.175153 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.175703 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.176760 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.177264 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.177372 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.177382 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.177567 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.177621 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:01:53.177940 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513ffdb3f47142fdb5edd045d23d7998" candidate_uuid: "222a77be2f6d470db42bfad8e6cb7174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef28d4b83a428abdc1f2265581b6a0" is_pre_election: true
I20260807 15:01:53.178092 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.174409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:36535 (local address 127.1.127.193:55239)
0807 15:01:53.174754 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.174770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.174859 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.175002 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.175003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.175005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.175076 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.175077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.175808 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.175809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.176646 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.176648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.177373 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.177374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.177463 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.177467 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.177931 (+ 464us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.177988 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":48,"thread_start_us":131,"threads_started":1}
I20260807 15:01:53.178136 3807 raft_consensus.cc:2454] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222a77be2f6d470db42bfad8e6cb7174 in term 0.
I20260807 15:01:53.178478 3451 leader_election.cc:302] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222a77be2f6d470db42bfad8e6cb7174, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:53.178506 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513ffdb3f47142fdb5edd045d23d7998" candidate_uuid: "222a77be2f6d470db42bfad8e6cb7174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd" is_pre_election: true
I20260807 15:01:53.178633 3896 raft_consensus.cc:2781] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:53.178727 3896 raft_consensus.cc:482] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:53.178768 3896 raft_consensus.cc:3037] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.178789 3673 raft_consensus.cc:2454] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222a77be2f6d470db42bfad8e6cb7174 in term 0.
I20260807 15:01:53.179679 3896 raft_consensus.cc:504] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.179849 3896 leader_election.cc:288] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [CANDIDATE]: Term 1 election: Requested vote from peers 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:53.180035 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513ffdb3f47142fdb5edd045d23d7998" candidate_uuid: "222a77be2f6d470db42bfad8e6cb7174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef28d4b83a428abdc1f2265581b6a0"
I20260807 15:01:53.180080 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513ffdb3f47142fdb5edd045d23d7998" candidate_uuid: "222a77be2f6d470db42bfad8e6cb7174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd"
I20260807 15:01:53.180153 3807 raft_consensus.cc:3037] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.180168 3673 raft_consensus.cc:3037] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.180938 3673 raft_consensus.cc:2454] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222a77be2f6d470db42bfad8e6cb7174 in term 1.
I20260807 15:01:53.181094 3807 raft_consensus.cc:2454] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222a77be2f6d470db42bfad8e6cb7174 in term 1.
I20260807 15:01:53.181627 3452 leader_election.cc:302] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222a77be2f6d470db42bfad8e6cb7174, 5745462d6b01477f8418f5f872afbefd; no voters:
I20260807 15:01:53.181803 3896 raft_consensus.cc:2781] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:53.182031 3896 raft_consensus.cc:686] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 LEADER]: Becoming Leader. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Running, Role: LEADER
I20260807 15:01:53.182158 3896 consensus_queue.cc:237] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } }
I20260807 15:01:53.183218 3357 catalog_manager.cc:5495] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "222a77be2f6d470db42bfad8e6cb7174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:01:53 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114912, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:53.215781 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.207535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33227 (local address 127.0.0.1:34698)
0807 15:01:53.207733 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.207746 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.207817 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.208156 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.208157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.208499 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:53.208559 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.208561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.209273 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.209274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.209830 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.209831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.210207 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.210208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.210265 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.210984 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:53.210989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:53.212679 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:53.214445 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.214455 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.214457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.214785 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.215055 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:53.215055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:53.215056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:53.215123 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:53.215396 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:53.215405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:53.215482 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.215554 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.215663 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":128,"threads_started":1}
I20260807 15:01:53.215793 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.207657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.1.127.254:33227)
0807 15:01:53.207739 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.207741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.207838 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:01:53.207869 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.207875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.207886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.207965 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:53.208683 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.209184 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.209912 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.210127 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.210183 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.212784 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:53.212786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:53.212786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:53.212793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:53.214340 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.214879 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.214880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.214880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.214938 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:53.215188 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:53.215188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:53.215189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:53.215304 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:53.215357 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.215583 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.215712 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":42}
I20260807 15:01:53.218216 3357 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:34698:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:01:53.218462 3357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:53.220296 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 48fc06e2b98d4a4da6e65af11c18a57e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1e174f2a7dd4dd1a8e303352c36670f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:53.220314 3796 tablet_service.cc:1467] Processing CreateTablet for tablet 48fc06e2b98d4a4da6e65af11c18a57e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1e174f2a7dd4dd1a8e303352c36670f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:53.220314 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 48fc06e2b98d4a4da6e65af11c18a57e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1e174f2a7dd4dd1a8e303352c36670f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:01:53.220499 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fc06e2b98d4a4da6e65af11c18a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.220505 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fc06e2b98d4a4da6e65af11c18a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.220512 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fc06e2b98d4a4da6e65af11c18a57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.221704 3892 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: Bootstrap starting.
I20260807 15:01:53.222147 3891 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: Bootstrap starting.
I20260807 15:01:53.222234 3892 tablet_bootstrap.cc:654] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.222231 3890 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: Bootstrap starting.
I20260807 15:01:53.222891 3890 tablet_bootstrap.cc:654] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.223016 3891 tablet_bootstrap.cc:654] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.223248 3892 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: No bootstrap required, opened a new log
I20260807 15:01:53.223304 3892 ts_tablet_manager.cc:1397] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:53.223474 3892 raft_consensus.cc:348] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.223578 3892 raft_consensus.cc:374] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.223606 3892 raft_consensus.cc:729] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.223663 3892 consensus_queue.cc:260] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.223773 3892 ts_tablet_manager.cc:1428] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.223785 3890 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: No bootstrap required, opened a new log
I20260807 15:01:53.223837 3890 ts_tablet_manager.cc:1397] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:53.223953 3896 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.224010 3890 raft_consensus.cc:348] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.224066 3896 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.224095 3890 raft_consensus.cc:374] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.224131 3890 raft_consensus.cc:729] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.224191 3890 consensus_queue.cc:260] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.224305 3890 ts_tablet_manager.cc:1428] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.224472 3897 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.224576 3897 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.224620 3891 tablet_bootstrap.cc:492] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: No bootstrap required, opened a new log
I20260807 15:01:53.224670 3891 ts_tablet_manager.cc:1397] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:01:53.224846 3891 raft_consensus.cc:348] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.224970 3891 raft_consensus.cc:374] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.225008 3891 raft_consensus.cc:729] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.225066 3891 consensus_queue.cc:260] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.225183 3891 ts_tablet_manager.cc:1428] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:01:53.225265 3898 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.225365 3898 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.232558 3897 raft_consensus.cc:482] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:53.232693 3897 raft_consensus.cc:504] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.233126 3897 leader_election.cc:288] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
W20260807 15:01:53.236325 3572 tablet.cc:2369] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:53.237600 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.233321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:32865 (local address 127.1.127.195:49245)
0807 15:01:53.233388 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.233401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.233446 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.234003 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.234004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.234006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.234085 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.234087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.235054 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.235056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.236487 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.236491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.237300 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.237301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.237396 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.237400 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.237447 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.237489 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":58}
I20260807 15:01:53.237913 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.233816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:49245 (local address 127.1.127.193:32865)
0807 15:01:53.233865 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.233867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.233877 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:53.233907 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.233908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.233918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.233982 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.234164 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.234946 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.236614 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.237186 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.237257 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.237267 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.237802 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.237842 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:01:53.237964 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.233829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:41463 (local address 127.1.127.194:36535)
0807 15:01:53.233874 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.233876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.233884 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:01:53.233910 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.233910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.233918 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.233981 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.234374 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.235090 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.236937 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.237500 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.237569 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.237577 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.237819 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.237878 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:01:53.238222 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.233455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:36535 (local address 127.1.127.195:41463)
0807 15:01:53.233762 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.233774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.233837 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.234167 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.234168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.234170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.234229 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.234230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.235161 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.235163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.236821 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.236825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.237612 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.237614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.237693 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.237698 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.238051 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.238107 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":194,"threads_started":1}
I20260807 15:01:53.238200 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fc06e2b98d4a4da6e65af11c18a57e" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174" is_pre_election: true
I20260807 15:01:53.238322 3522 raft_consensus.cc:2454] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:53.238623 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fc06e2b98d4a4da6e65af11c18a57e" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd" is_pre_election: true
I20260807 15:01:53.238734 3673 raft_consensus.cc:2454] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:53.238925 3749 leader_election.cc:302] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5745462d6b01477f8418f5f872afbefd, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:53.239110 3897 raft_consensus.cc:2781] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:53.239156 3897 raft_consensus.cc:482] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:53.239187 3897 raft_consensus.cc:3037] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.240149 3897 raft_consensus.cc:504] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.240303 3897 leader_election.cc:288] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Requested vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:53.240482 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fc06e2b98d4a4da6e65af11c18a57e" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174"
I20260807 15:01:53.240522 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fc06e2b98d4a4da6e65af11c18a57e" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd"
I20260807 15:01:53.240571 3522 raft_consensus.cc:3037] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.240586 3673 raft_consensus.cc:3037] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.241426 3522 raft_consensus.cc:2454] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:53.241426 3673 raft_consensus.cc:2454] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:53.241640 3749 leader_election.cc:302] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222a77be2f6d470db42bfad8e6cb7174, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:53.241792 3897 raft_consensus.cc:2781] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:53.241986 3897 raft_consensus.cc:686] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Running, Role: LEADER
I20260807 15:01:53.242111 3897 consensus_queue.cc:237] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.243289 3912 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: TxnStatusTablet state changed. Reason: New leader 81ef28d4b83a428abdc1f2265581b6a0. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.243393 3912 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:53.243760 3914 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:01:53.243919 3914 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:01:53.243930 3357 catalog_manager.cc:5495] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } health_report { overall_health: HEALTHY } } }
I20260807 15:01:53.251751 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.247642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.1.127.195:43455)
0807 15:01:53.248289 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.248291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.248301 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:53.248340 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.248340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.248404 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.248491 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.248677 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.249375 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.250564 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.250965 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.251064 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.251312 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.251454 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.251552 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.251623 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.251656 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618}
I20260807 15:01:53.251806 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.247574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.0.0.1:39126)
0807 15:01:53.247663 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.247674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.247723 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.248523 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.248524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.248526 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:53.248585 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.248587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.249479 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.249481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.250446 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.250450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.251076 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.251077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.251159 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.251189 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.251541 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.251555 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.251619 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.251689 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 15:01:53.254102 3522 raft_consensus.cc:1261] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.254104 3673 raft_consensus.cc:1261] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.254468 3897 consensus_queue.cc:1035] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.254601 3912 consensus_queue.cc:1035] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.255894 3896 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: TxnStatusTablet state changed. Reason: New leader 81ef28d4b83a428abdc1f2265581b6a0. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.255996 3896 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.256259 3898 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: TxnStatusTablet state changed. Reason: New leader 81ef28d4b83a428abdc1f2265581b6a0. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.256387 3898 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.257292 3897 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.257382 3897 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:53.257594 3912 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.257659 3912 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 81ef28d4b83a428abdc1f2265581b6a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:01:53.258087 3918 mvcc.cc:205] Tried to move back new op lower bound from 7315926684685881344 to 7315926684640493568. Current Snapshot: MvccSnapshot[applied={T|T < 7315926684685881344}]
I20260807 15:01:53.258181 3896 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.258247 3896 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 222a77be2f6d470db42bfad8e6cb7174: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.259250 3898 tablet_replica.cc:439] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } } }
I20260807 15:01:53.259348 3898 tablet_replica.cc:442] T 48fc06e2b98d4a4da6e65af11c18a57e P 5745462d6b01477f8418f5f872afbefd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:01:53.263368 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.260338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46650 (local address 127.1.127.193:32865)
0807 15:01:53.260409 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.260411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.260420 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:01:53.260457 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.260457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.260477 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.260548 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.260640 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.261539 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.262379 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.262790 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.262867 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.263013 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.263146 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.263229 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.263247 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.263287 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:01:53.269390 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.266161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.1.127.193:37637)
0807 15:01:53.266498 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.266510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.266662 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.266868 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.266868 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.266870 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:53.266926 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.266928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.267818 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.267819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.268420 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.268423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.268911 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.268912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.268975 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.268993 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.269236 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.269238 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.269277 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.269319 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":265,"threads_started":1}
I20260807 15:01:53.269384 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.266230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37637 (local address 127.1.127.195:43455)
0807 15:01:53.266307 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.266310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.266664 (+ 354us) server_negotiation.cc:407] Connection header received
0807 15:01:53.266789 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.266790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.266802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.266857 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.266989 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.267730 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.268508 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.268815 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.268888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.269118 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.269166 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.269227 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.269281 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.269303 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:01:53.272598 3673 raft_consensus.cc:1261] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Refusing update from remote peer 222a77be2f6d470db42bfad8e6cb7174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.272708 3807 raft_consensus.cc:1261] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Refusing update from remote peer 222a77be2f6d470db42bfad8e6cb7174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.273016 3896 consensus_queue.cc:1035] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.273190 3900 consensus_queue.cc:1035] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.275902 3920 mvcc.cc:205] Tried to move back new op lower bound from 7315926684762742784 to 7315926684395143168. Current Snapshot: MvccSnapshot[applied={T|T < 7315926684762742784}]
I20260807 15:01:53.284102 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.280967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:32865 (local address 127.1.127.195:50539)
0807 15:01:53.281075 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.281088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.281149 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.281280 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.281281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.281282 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:53.281340 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.281341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.282218 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.282219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.283186 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.283188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.283647 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.283648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.283715 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.283731 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.283910 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.283912 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.284006 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.284045 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 15:01:53.284125 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.280995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:50539 (local address 127.1.127.193:32865)
0807 15:01:53.281035 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.281037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.281134 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:01:53.281202 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.281202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.281215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.281279 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.281380 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.282130 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.283276 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.283568 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.283640 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.283817 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.283851 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.283903 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.284013 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.284036 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:01:53.294170 3355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:53.294310 3355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:53.295246 3355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Sending DeleteTablet for 3 replicas of tablet 513ffdb3f47142fdb5edd045d23d7998
I20260807 15:01:53.295589 3497 tablet_service.cc:1514] Processing DeleteTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:53 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46646
I20260807 15:01:53.295650 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:53 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:45098
I20260807 15:01:53.295776 3796 tablet_service.cc:1514] Processing DeleteTablet for tablet 513ffdb3f47142fdb5edd045d23d7998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:53 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39110
I20260807 15:01:53.296305 3355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:53.296460 3355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:53.298326 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 (DEFAULT_TABLE table=test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.298370 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 (DEFAULT_TABLE table=test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.298503 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c7cb72fb9c4261ba7965cc383ce464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.298558 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c7cb72fb9c4261ba7965cc383ce464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.300028 3892 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Bootstrap starting.
I20260807 15:01:53.300201 3891 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Bootstrap starting.
I20260807 15:01:53.300489 3892 tablet_bootstrap.cc:654] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.300704 3891 tablet_bootstrap.cc:654] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.301468 3892 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: No bootstrap required, opened a new log
I20260807 15:01:53.301487 3796 tablet_service.cc:1467] Processing CreateTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 (DEFAULT_TABLE table=test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:53.301525 3892 ts_tablet_manager.cc:1397] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:53.301620 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51c7cb72fb9c4261ba7965cc383ce464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:53.301698 3892 raft_consensus.cc:348] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.301769 3892 raft_consensus.cc:374] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.301792 3892 raft_consensus.cc:729] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.301836 3892 consensus_queue.cc:260] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.301942 3892 ts_tablet_manager.cc:1428] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.302879 3891 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: No bootstrap required, opened a new log
I20260807 15:01:53.302970 3891 ts_tablet_manager.cc:1397] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:01:53.303150 3891 raft_consensus.cc:348] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.303251 3891 raft_consensus.cc:374] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.303241 3890 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Bootstrap starting.
I20260807 15:01:53.303275 3891 raft_consensus.cc:729] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.303333 3891 consensus_queue.cc:260] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.303467 3891 ts_tablet_manager.cc:1428] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:53.303726 3890 tablet_bootstrap.cc:654] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:53.306821 3939 tablet_replica.cc:330] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: stopping tablet replica
I20260807 15:01:53.306998 3939 raft_consensus.cc:2229] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:53.307085 3939 raft_consensus.cc:2258] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:53.307600 3939 ts_tablet_manager.cc:1905] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:53.309733 3939 ts_tablet_manager.cc:1918] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:53.309821 3939 log.cc:1196] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/513ffdb3f47142fdb5edd045d23d7998
I20260807 15:01:53.310109 3939 ts_tablet_manager.cc:1939] T 513ffdb3f47142fdb5edd045d23d7998 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting consensus metadata
I20260807 15:01:53.310595 3341 catalog_manager.cc:4841] TS 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455): tablet 513ffdb3f47142fdb5edd045d23d7998 (table test-table [id=ea15216e107c4265b1cc4ef8b7756a31]) successfully deleted
I20260807 15:01:53.310811 3937 tablet_replica.cc:330] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: stopping tablet replica
I20260807 15:01:53.310997 3937 raft_consensus.cc:2229] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:53.311152 3937 raft_consensus.cc:2258] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:53.311398 3938 tablet_replica.cc:330] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: stopping tablet replica
I20260807 15:01:53.311544 3938 raft_consensus.cc:2229] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:53.311622 3938 raft_consensus.cc:2258] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:53.311671 3937 ts_tablet_manager.cc:1905] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:53.312060 3938 ts_tablet_manager.cc:1905] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:53.313486 3937 ts_tablet_manager.cc:1918] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:53.313584 3937 log.cc:1196] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/513ffdb3f47142fdb5edd045d23d7998
I20260807 15:01:53.313711 3938 ts_tablet_manager.cc:1918] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:53.313835 3938 log.cc:1196] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/513ffdb3f47142fdb5edd045d23d7998
I20260807 15:01:53.313859 3937 ts_tablet_manager.cc:1939] T 513ffdb3f47142fdb5edd045d23d7998 P 222a77be2f6d470db42bfad8e6cb7174: Deleting consensus metadata
I20260807 15:01:53.314174 3938 ts_tablet_manager.cc:1939] T 513ffdb3f47142fdb5edd045d23d7998 P 5745462d6b01477f8418f5f872afbefd: Deleting consensus metadata
I20260807 15:01:53.314580 3343 catalog_manager.cc:4841] TS 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865): tablet 513ffdb3f47142fdb5edd045d23d7998 (table test-table [id=ea15216e107c4265b1cc4ef8b7756a31]) successfully deleted
I20260807 15:01:53.314713 3343 catalog_manager.cc:4841] TS 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535): tablet 513ffdb3f47142fdb5edd045d23d7998 (table test-table [id=ea15216e107c4265b1cc4ef8b7756a31]) successfully deleted
I20260807 15:01:53.315039 3890 tablet_bootstrap.cc:492] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: No bootstrap required, opened a new log
I20260807 15:01:53.315101 3890 ts_tablet_manager.cc:1397] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20260807 15:01:53.315275 3890 raft_consensus.cc:348] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.315358 3890 raft_consensus.cc:374] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:53.315378 3890 raft_consensus.cc:729] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Initialized, Role: FOLLOWER
I20260807 15:01:53.315430 3890 consensus_queue.cc:260] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.315524 3890 ts_tablet_manager.cc:1428] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260807 15:01:53.346561 3868 tablet.cc:2369] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:01:53.357520 3898 raft_consensus.cc:482] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:53.357671 3898 raft_consensus.cc:504] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.358104 3898 leader_election.cc:288] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455)
I20260807 15:01:53.362591 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.358311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:32865 (local address 127.1.127.194:34537)
0807 15:01:53.358372 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.358383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.358429 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.358883 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.358884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.358887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.358962 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.358964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.359984 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.359985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.361301 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.361304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.362260 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.362261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.362360 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.362364 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.362430 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.362478 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:01:53.362850 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.358591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:34537 (local address 127.1.127.193:32865)
0807 15:01:53.358641 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.358643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.358654 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:01:53.358724 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.358725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.358736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.358806 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.359064 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.359847 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.361430 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.362100 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.362181 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.362193 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.362735 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.362779 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:01:53.363404 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.358586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:59039 (local address 127.1.127.195:43455)
0807 15:01:53.358634 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.358636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.358905 (+ 269us) server_negotiation.cc:407] Connection header received
0807 15:01:53.359025 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.359026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.359038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.359068 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:01:53.359812 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.360524 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.361827 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.362407 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.362465 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.362476 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.363238 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.363298 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:01:53.363391 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c7cb72fb9c4261ba7965cc383ce464" candidate_uuid: "5745462d6b01477f8418f5f872afbefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174" is_pre_election: true
I20260807 15:01:53.363498 3522 raft_consensus.cc:2454] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5745462d6b01477f8418f5f872afbefd in term 0.
I20260807 15:01:53.363736 3599 leader_election.cc:302] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222a77be2f6d470db42bfad8e6cb7174, 5745462d6b01477f8418f5f872afbefd; no voters:
I20260807 15:01:53.363903 3898 raft_consensus.cc:2781] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:53.363965 3898 raft_consensus.cc:482] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:53.363988 3898 raft_consensus.cc:3037] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.364410 3943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.358356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.1.127.194:59039)
0807 15:01:53.358724 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.358734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.358937 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.359133 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.359134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.359136 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:01:53.359706 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.359709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.360621 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.360622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.361713 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.361716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.362603 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.362604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.363121 (+ 517us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.363129 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.363797 (+ 668us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.363898 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":64,"threads_started":1}
I20260807 15:01:53.364795 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c7cb72fb9c4261ba7965cc383ce464" candidate_uuid: "5745462d6b01477f8418f5f872afbefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef28d4b83a428abdc1f2265581b6a0" is_pre_election: true
I20260807 15:01:53.364887 3807 raft_consensus.cc:2454] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5745462d6b01477f8418f5f872afbefd in term 0.
I20260807 15:01:53.364986 3898 raft_consensus.cc:504] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.365136 3898 leader_election.cc:288] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [CANDIDATE]: Term 1 election: Requested vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455)
I20260807 15:01:53.365334 3807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c7cb72fb9c4261ba7965cc383ce464" candidate_uuid: "5745462d6b01477f8418f5f872afbefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81ef28d4b83a428abdc1f2265581b6a0"
I20260807 15:01:53.365357 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51c7cb72fb9c4261ba7965cc383ce464" candidate_uuid: "5745462d6b01477f8418f5f872afbefd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174"
I20260807 15:01:53.365410 3807 raft_consensus.cc:3037] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.365437 3522 raft_consensus.cc:3037] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:53.366210 3807 raft_consensus.cc:2454] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5745462d6b01477f8418f5f872afbefd in term 1.
I20260807 15:01:53.366304 3522 raft_consensus.cc:2454] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5745462d6b01477f8418f5f872afbefd in term 1.
I20260807 15:01:53.366478 3598 leader_election.cc:302] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5745462d6b01477f8418f5f872afbefd, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:53.366595 3898 raft_consensus.cc:2781] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:53.366791 3898 raft_consensus.cc:686] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 1 LEADER]: Becoming Leader. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Running, Role: LEADER
I20260807 15:01:53.366892 3898 consensus_queue.cc:237] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:53.367678 3355 catalog_manager.cc:5495] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5745462d6b01477f8418f5f872afbefd (127.1.127.194). New cstate: current_term: 1 leader_uuid: "5745462d6b01477f8418f5f872afbefd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } health_report { overall_health: UNKNOWN } } }
I20260807 15:01:53.387238 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.383896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.1.127.194:36535)
0807 15:01:53.383941 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.383944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.383954 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:01:53.384003 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.384004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.384034 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.384136 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.384307 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.385094 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.386184 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.386487 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.386564 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.386786 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.386933 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.387013 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.387082 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.387148 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:01:53.393893 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.390801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:33141 (local address 127.1.127.195:43455)
0807 15:01:53.390847 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.390849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.390858 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:01:53.390883 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.390884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.390895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.390960 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.391131 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.391873 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.392979 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.393285 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.393368 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.393602 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.393642 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.393704 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.393785 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.393809 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:01:53.393914 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.390502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.1.127.194:33141)
0807 15:01:53.390594 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.390607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.390650 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.390984 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.390985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.390987 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:53.391049 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.391051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.391981 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.391982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.392869 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.392872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.393391 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.393393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.393473 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.393494 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.393724 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.393727 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.393773 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.393824 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:01:53.397011 3807 raft_consensus.cc:1261] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Refusing update from remote peer 5745462d6b01477f8418f5f872afbefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.397307 3944 consensus_queue.cc:1035] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.397557 3522 raft_consensus.cc:1261] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Refusing update from remote peer 5745462d6b01477f8418f5f872afbefd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:53.398551 3944 consensus_queue.cc:1035] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [LEADER]: Connected to new peer: Peer: permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:53.402303 3917 mvcc.cc:205] Tried to move back new op lower bound from 7315926685272375296 to 7315926685151371264. Current Snapshot: MvccSnapshot[applied={T|T < 7315926685272375296}]
I20260807 15:01:53.410331 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.407280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:36535 (local address 127.1.127.195:48945)
0807 15:01:53.407347 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:53.407361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:53.407402 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:53.407564 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:53.407565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:53.407566 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:53.407613 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.407615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.408555 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.408557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.409369 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:53.409373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.409904 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.409905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:53.409962 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.409968 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.410147 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.410149 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:01:53.410188 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 15:01:53.410244 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:01:53.410892 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:53.407370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:48945 (local address 127.1.127.194:36535)
0807 15:01:53.407409 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:53.407411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:53.407423 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:01:53.407448 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:53.407449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:53.407459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:53.407512 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:53.407702 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.408435 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.409475 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:53.409789 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:53.409866 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:53.410045 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:53.410083 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:53.410125 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:53.410770 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 15:01:53.410802 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:01:53.411675 3920 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:01:54.939116 3355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:54.939229 3355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:54.940529 3355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Sending DeleteTablet for 3 replicas of tablet 51c7cb72fb9c4261ba7965cc383ce464
I20260807 15:01:54.940866 3497 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46646
I20260807 15:01:54.940907 3796 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39110
I20260807 15:01:54.941112 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 51c7cb72fb9c4261ba7965cc383ce464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:45098
I20260807 15:01:54.941361 3975 tablet_replica.cc:330] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: stopping tablet replica
I20260807 15:01:54.941368 3977 tablet_replica.cc:330] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: stopping tablet replica
I20260807 15:01:54.941444 3977 raft_consensus.cc:2229] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:54.941444 3975 raft_consensus.cc:2229] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:54.941502 3975 raft_consensus.cc:2258] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:54.941562 3977 raft_consensus.cc:2258] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:54.941802 3975 ts_tablet_manager.cc:1905] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:54.941819 3977 ts_tablet_manager.cc:1905] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:54.943306 3976 tablet_replica.cc:330] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: stopping tablet replica
I20260807 15:01:54.943411 3976 raft_consensus.cc:2229] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:54.943472 3976 raft_consensus.cc:2258] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:54.943624 3975 ts_tablet_manager.cc:1918] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:01:54.943699 3975 log.cc:1196] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/51c7cb72fb9c4261ba7965cc383ce464
I20260807 15:01:54.943956 3975 ts_tablet_manager.cc:1939] T 51c7cb72fb9c4261ba7965cc383ce464 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting consensus metadata
I20260807 15:01:54.944051 3977 ts_tablet_manager.cc:1918] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:01:54.944130 3977 log.cc:1196] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/51c7cb72fb9c4261ba7965cc383ce464
I20260807 15:01:54.944375 3977 ts_tablet_manager.cc:1939] T 51c7cb72fb9c4261ba7965cc383ce464 P 5745462d6b01477f8418f5f872afbefd: Deleting consensus metadata
I20260807 15:01:54.944552 3976 ts_tablet_manager.cc:1905] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:54.945080 3343 catalog_manager.cc:4841] TS 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535): tablet 51c7cb72fb9c4261ba7965cc383ce464 (table test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]) successfully deleted
I20260807 15:01:54.945101 3341 catalog_manager.cc:4841] TS 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455): tablet 51c7cb72fb9c4261ba7965cc383ce464 (table test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]) successfully deleted
I20260807 15:01:54.946390 3976 ts_tablet_manager.cc:1918] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:01:54.946481 3976 log.cc:1196] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/51c7cb72fb9c4261ba7965cc383ce464
I20260807 15:01:54.946784 3976 ts_tablet_manager.cc:1939] T 51c7cb72fb9c4261ba7965cc383ce464 P 222a77be2f6d470db42bfad8e6cb7174: Deleting consensus metadata
I20260807 15:01:54.947360 3343 catalog_manager.cc:4841] TS 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865): tablet 51c7cb72fb9c4261ba7965cc383ce464 (table test-table [id=c6e0675d657c4ace9bb33fda0dcf080b]) successfully deleted
Aug 07 15:01:54 dist-test-slave-8tdl krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:01:54 dist-test-slave-8tdl krb5kdc[3310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:54.971668 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:54.961547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.1.127.254:33227)
0807 15:01:54.961756 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:54.961758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:54.961861 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:01:54.961931 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:54.961931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:54.961943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:54.962137 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:54.962940 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:54.963602 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:54.964587 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:54.964803 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:54.964895 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:54.968139 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:54.968144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:54.968145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:54.968154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:54.970044 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:54.970516 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:54.970517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:54.970518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:54.970585 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:54.970945 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:54.970945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:54.970946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:54.971086 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:54.971179 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:54.971497 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 15:01:54.971590 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":133,"threads_started":1}
I20260807 15:01:54.973507 3355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:54.973711 3355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:54.975643 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 452504a882294d6caa04676caece9018 (DEFAULT_TABLE table=test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:54.975672 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 452504a882294d6caa04676caece9018 (DEFAULT_TABLE table=test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:54.975693 3796 tablet_service.cc:1467] Processing CreateTablet for tablet 452504a882294d6caa04676caece9018 (DEFAULT_TABLE table=test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:54.975832 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452504a882294d6caa04676caece9018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:54.975832 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452504a882294d6caa04676caece9018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:54.975835 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452504a882294d6caa04676caece9018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:54.977028 3987 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Bootstrap starting.
I20260807 15:01:54.977344 3988 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Bootstrap starting.
I20260807 15:01:54.977578 3989 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Bootstrap starting.
I20260807 15:01:54.977626 3987 tablet_bootstrap.cc:654] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:54.977728 3988 tablet_bootstrap.cc:654] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:54.978031 3989 tablet_bootstrap.cc:654] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:54.978426 3987 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: No bootstrap required, opened a new log
I20260807 15:01:54.978502 3987 ts_tablet_manager.cc:1397] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:54.978629 3988 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: No bootstrap required, opened a new log
I20260807 15:01:54.978720 3988 ts_tablet_manager.cc:1397] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:54.978657 3987 raft_consensus.cc:348] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.978765 3987 raft_consensus.cc:374] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:54.978767 3989 tablet_bootstrap.cc:492] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: No bootstrap required, opened a new log
I20260807 15:01:54.978793 3987 raft_consensus.cc:729] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Initialized, Role: FOLLOWER
I20260807 15:01:54.978816 3989 ts_tablet_manager.cc:1397] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:54.978861 3987 consensus_queue.cc:260] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.978885 3988 raft_consensus.cc:348] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.978965 3988 raft_consensus.cc:374] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:54.978927 3989 raft_consensus.cc:348] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.978992 3988 raft_consensus.cc:729] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Initialized, Role: FOLLOWER
I20260807 15:01:54.978992 3989 raft_consensus.cc:374] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:54.979014 3989 raft_consensus.cc:729] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Initialized, Role: FOLLOWER
I20260807 15:01:54.979048 3988 consensus_queue.cc:260] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.979067 3989 consensus_queue.cc:260] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:54.979164 3989 ts_tablet_manager.cc:1428] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:54.979158 3988 ts_tablet_manager.cc:1428] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:54.979101 3987 ts_tablet_manager.cc:1428] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:55.010526 3969 raft_consensus.cc:482] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:55.010660 3969 raft_consensus.cc:504] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.011014 3969 leader_election.cc:288] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:55.011305 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452504a882294d6caa04676caece9018" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174" is_pre_election: true
I20260807 15:01:55.011319 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452504a882294d6caa04676caece9018" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd" is_pre_election: true
I20260807 15:01:55.011420 3522 raft_consensus.cc:2454] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:55.011443 3673 raft_consensus.cc:2454] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:55.011639 3749 leader_election.cc:302] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5745462d6b01477f8418f5f872afbefd, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:55.011775 3969 raft_consensus.cc:2781] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:55.011854 3969 raft_consensus.cc:482] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:55.011893 3969 raft_consensus.cc:3037] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.012771 3969 raft_consensus.cc:504] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.012961 3969 leader_election.cc:288] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Requested vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:55.013293 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452504a882294d6caa04676caece9018" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174"
I20260807 15:01:55.013293 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452504a882294d6caa04676caece9018" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd"
I20260807 15:01:55.013409 3522 raft_consensus.cc:3037] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.013409 3673 raft_consensus.cc:3037] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.014420 3522 raft_consensus.cc:2454] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:55.014426 3673 raft_consensus.cc:2454] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:55.014643 3749 leader_election.cc:302] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222a77be2f6d470db42bfad8e6cb7174, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:55.014854 3969 raft_consensus.cc:2781] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:55.014953 3969 raft_consensus.cc:686] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Running, Role: LEADER
I20260807 15:01:55.015115 3969 consensus_queue.cc:237] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.015921 3355 catalog_manager.cc:5495] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } health_report { overall_health: HEALTHY } } }
I20260807 15:01:55.031517 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:55.028855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.1.127.195:43455)
0807 15:01:55.029026 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:55.029028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:55.029039 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:01:55.029062 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:55.029063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:55.029074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:55.029144 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:55.029301 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.029883 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.030774 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.031054 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.031119 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:55.031247 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:55.031275 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:55.031310 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:55.031371 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 15:01:55.031391 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":116,"threads_started":1}
I20260807 15:01:55.036672 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:55.034301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:52829 (local address 127.1.127.195:43455)
0807 15:01:55.034360 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:55.034363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:55.034371 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:01:55.034437 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:55.034438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:55.034450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:55.034498 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:01:55.034623 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.035144 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.035878 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.036084 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.036185 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:55.036378 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:01:55.036410 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:01:55.036470 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:55.036535 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 15:01:55.036558 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":42}
I20260807 15:01:55.036688 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:55.034141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:43455 (local address 127.1.127.195:52829)
0807 15:01:55.034296 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:55.034306 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:55.034377 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:55.034499 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:55.034500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:55.034501 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:01:55.034550 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:55.034551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.035234 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.035235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:55.035791 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:55.035793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:55.036205 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:55.036206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:55.036261 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:55.036266 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:01:55.036485 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:01:55.036487 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:01:55.036531 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:01:55.036596 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260807 15:01:55.039366 3522 raft_consensus.cc:1261] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:55.039399 3673 raft_consensus.cc:1261] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:55.039609 3969 consensus_queue.cc:1035] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:55.039723 3930 consensus_queue.cc:1035] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:55.052151 3358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:55.052434 3358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:55.053258 3358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Sending DeleteTablet for 3 replicas of tablet 452504a882294d6caa04676caece9018
I20260807 15:01:55.053565 3795 tablet_service.cc:1514] Processing DeleteTablet for tablet 452504a882294d6caa04676caece9018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39110
I20260807 15:01:55.053666 3975 tablet_replica.cc:330] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: stopping tablet replica
I20260807 15:01:55.053640 3497 tablet_service.cc:1514] Processing DeleteTablet for tablet 452504a882294d6caa04676caece9018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46646
I20260807 15:01:55.053721 3975 raft_consensus.cc:2229] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:55.053741 3976 tablet_replica.cc:330] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: stopping tablet replica
I20260807 15:01:55.053834 3975 raft_consensus.cc:2258] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:55.053952 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 452504a882294d6caa04676caece9018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:45098
I20260807 15:01:55.054049 3976 raft_consensus.cc:2229] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:55.054106 3976 raft_consensus.cc:2258] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:55.054031 3358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:01:55.054232 3358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:01:55.054382 3975 ts_tablet_manager.cc:1905] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:55.054427 3977 tablet_replica.cc:330] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: stopping tablet replica
I20260807 15:01:55.054481 3976 ts_tablet_manager.cc:1905] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:55.054500 3977 raft_consensus.cc:2229] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:55.054548 3977 raft_consensus.cc:2258] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:55.056324 3795 tablet_service.cc:1467] Processing CreateTablet for tablet 0175e11870524ec78754dbf082fbc6e0 (DEFAULT_TABLE table=test-table [id=6a263dab53a94236bbfb6bc8e4776f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:55.056555 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 0175e11870524ec78754dbf082fbc6e0 (DEFAULT_TABLE table=test-table [id=6a263dab53a94236bbfb6bc8e4776f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:55.056690 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0175e11870524ec78754dbf082fbc6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:55.056782 3977 ts_tablet_manager.cc:1905] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:55.058149 3988 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Bootstrap starting.
I20260807 15:01:55.058517 3988 tablet_bootstrap.cc:654] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:55.058846 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 0175e11870524ec78754dbf082fbc6e0 (DEFAULT_TABLE table=test-table [id=6a263dab53a94236bbfb6bc8e4776f5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:01:55.059185 3977 ts_tablet_manager.cc:1918] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:55.059190 3988 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: No bootstrap required, opened a new log
I20260807 15:01:55.059274 3977 log.cc:1196] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/452504a882294d6caa04676caece9018
I20260807 15:01:55.059285 3988 ts_tablet_manager.cc:1397] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:01:55.059418 3988 raft_consensus.cc:348] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.059496 3988 raft_consensus.cc:374] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:55.059520 3977 ts_tablet_manager.cc:1939] T 452504a882294d6caa04676caece9018 P 5745462d6b01477f8418f5f872afbefd: Deleting consensus metadata
I20260807 15:01:55.059530 3988 raft_consensus.cc:729] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5745462d6b01477f8418f5f872afbefd, State: Initialized, Role: FOLLOWER
I20260807 15:01:55.059602 3988 consensus_queue.cc:260] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.059701 3988 ts_tablet_manager.cc:1428] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:55.059875 3976 ts_tablet_manager.cc:1918] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:55.059962 3976 log.cc:1196] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/452504a882294d6caa04676caece9018
I20260807 15:01:55.060017 3343 catalog_manager.cc:4841] TS 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535): tablet 452504a882294d6caa04676caece9018 (table test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]) successfully deleted
I20260807 15:01:55.060186 3976 ts_tablet_manager.cc:1939] T 452504a882294d6caa04676caece9018 P 222a77be2f6d470db42bfad8e6cb7174: Deleting consensus metadata
I20260807 15:01:55.060389 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0175e11870524ec78754dbf082fbc6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:55.060628 3343 catalog_manager.cc:4841] TS 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865): tablet 452504a882294d6caa04676caece9018 (table test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]) successfully deleted
I20260807 15:01:55.061635 3987 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174: Bootstrap starting.
I20260807 15:01:55.061992 3987 tablet_bootstrap.cc:654] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:55.062119 3975 ts_tablet_manager.cc:1918] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:01:55.062191 3975 log.cc:1196] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/452504a882294d6caa04676caece9018
I20260807 15:01:55.062402 3975 ts_tablet_manager.cc:1939] T 452504a882294d6caa04676caece9018 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting consensus metadata
I20260807 15:01:55.062570 3795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0175e11870524ec78754dbf082fbc6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:55.062839 3341 catalog_manager.cc:4841] TS 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455): tablet 452504a882294d6caa04676caece9018 (table test-table [id=4442d2c1d5ab498b9425f63c443c0ef2]) successfully deleted
I20260807 15:01:55.063216 3987 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174: No bootstrap required, opened a new log
I20260807 15:01:55.063279 3987 ts_tablet_manager.cc:1397] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:01:55.063416 3987 raft_consensus.cc:348] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.063508 3987 raft_consensus.cc:374] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:55.063534 3987 raft_consensus.cc:729] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222a77be2f6d470db42bfad8e6cb7174, State: Initialized, Role: FOLLOWER
I20260807 15:01:55.063592 3987 consensus_queue.cc:260] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.063692 3987 ts_tablet_manager.cc:1428] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:01:55.064168 3989 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Bootstrap starting.
I20260807 15:01:55.064543 3989 tablet_bootstrap.cc:654] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:55.065236 3989 tablet_bootstrap.cc:492] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: No bootstrap required, opened a new log
I20260807 15:01:55.065281 3989 ts_tablet_manager.cc:1397] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:55.065466 3989 raft_consensus.cc:348] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.065560 3989 raft_consensus.cc:374] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:55.065585 3989 raft_consensus.cc:729] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Initialized, Role: FOLLOWER
I20260807 15:01:55.065640 3989 consensus_queue.cc:260] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.065757 3989 ts_tablet_manager.cc:1428] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:55.151537 3969 raft_consensus.cc:482] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:01:55.151684 3969 raft_consensus.cc:504] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.151901 3969 leader_election.cc:288] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:55.152208 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0175e11870524ec78754dbf082fbc6e0" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd" is_pre_election: true
I20260807 15:01:55.152211 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0175e11870524ec78754dbf082fbc6e0" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174" is_pre_election: true
I20260807 15:01:55.152307 3673 raft_consensus.cc:2454] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:55.152329 3522 raft_consensus.cc:2454] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 0.
I20260807 15:01:55.152464 3749 leader_election.cc:302] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5745462d6b01477f8418f5f872afbefd, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:55.152547 3969 raft_consensus.cc:2781] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:01:55.152580 3969 raft_consensus.cc:482] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:01:55.152601 3969 raft_consensus.cc:3037] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.153575 3969 raft_consensus.cc:504] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.153740 3969 leader_election.cc:288] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Requested vote from peers 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865), 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535)
I20260807 15:01:55.154080 3673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0175e11870524ec78754dbf082fbc6e0" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5745462d6b01477f8418f5f872afbefd"
I20260807 15:01:55.154085 3522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0175e11870524ec78754dbf082fbc6e0" candidate_uuid: "81ef28d4b83a428abdc1f2265581b6a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222a77be2f6d470db42bfad8e6cb7174"
I20260807 15:01:55.154177 3522 raft_consensus.cc:3037] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.154175 3673 raft_consensus.cc:3037] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:55.154973 3673 raft_consensus.cc:2454] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:55.155081 3522 raft_consensus.cc:2454] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81ef28d4b83a428abdc1f2265581b6a0 in term 1.
I20260807 15:01:55.155233 3749 leader_election.cc:302] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5745462d6b01477f8418f5f872afbefd, 81ef28d4b83a428abdc1f2265581b6a0; no voters:
I20260807 15:01:55.155408 3969 raft_consensus.cc:2781] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:55.155483 3969 raft_consensus.cc:686] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 LEADER]: Becoming Leader. State: Replica: 81ef28d4b83a428abdc1f2265581b6a0, State: Running, Role: LEADER
I20260807 15:01:55.155529 3969 consensus_queue.cc:237] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } }
I20260807 15:01:55.156273 3358 catalog_manager.cc:5495] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "81ef28d4b83a428abdc1f2265581b6a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81ef28d4b83a428abdc1f2265581b6a0" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 43455 } health_report { overall_health: HEALTHY } } }
I20260807 15:01:55.184832 3522 raft_consensus.cc:1261] T 0175e11870524ec78754dbf082fbc6e0 P 222a77be2f6d470db42bfad8e6cb7174 [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:55.184831 3673 raft_consensus.cc:1261] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Refusing update from remote peer 81ef28d4b83a428abdc1f2265581b6a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:01:55.185087 3930 consensus_queue.cc:1035] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222a77be2f6d470db42bfad8e6cb7174" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:01:55.185202 3969 consensus_queue.cc:1035] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5745462d6b01477f8418f5f872afbefd" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:01:55.190567 3953 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:01:56.711721 3355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:56.711836 3355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:01:56.712913 3355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c212a87052c442481a7d40dfcbd9d0e: Sending DeleteTablet for 3 replicas of tablet 0175e11870524ec78754dbf082fbc6e0
I20260807 15:01:56.713220 3795 tablet_service.cc:1514] Processing DeleteTablet for tablet 0175e11870524ec78754dbf082fbc6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:56 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:39110
I20260807 15:01:56.713260 3497 tablet_service.cc:1514] Processing DeleteTablet for tablet 0175e11870524ec78754dbf082fbc6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:56 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46646
I20260807 15:01:56.713266 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 0175e11870524ec78754dbf082fbc6e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:01:56 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:45098
I20260807 15:01:56.713342 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 3433
I20260807 15:01:56.715502 4033 tablet_replica.cc:330] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: stopping tablet replica
I20260807 15:01:56.715595 4033 raft_consensus.cc:2229] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:01:56.715685 4033 raft_consensus.cc:2258] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:56.715993 4033 ts_tablet_manager.cc:1905] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:56.717257 4035 tablet_replica.cc:330] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: stopping tablet replica
I20260807 15:01:56.717357 4035 raft_consensus.cc:2229] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:01:56.717423 4035 raft_consensus.cc:2258] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:01:56.717641 4033 ts_tablet_manager.cc:1918] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:01:56.717684 4035 ts_tablet_manager.cc:1905] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:01:56.717723 4033 log.cc:1196] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/0175e11870524ec78754dbf082fbc6e0
I20260807 15:01:56.718015 4033 ts_tablet_manager.cc:1939] T 0175e11870524ec78754dbf082fbc6e0 P 81ef28d4b83a428abdc1f2265581b6a0: Deleting consensus metadata
I20260807 15:01:56.718413 3341 catalog_manager.cc:4841] TS 81ef28d4b83a428abdc1f2265581b6a0 (127.1.127.195:43455): tablet 0175e11870524ec78754dbf082fbc6e0 (table test-table [id=6a263dab53a94236bbfb6bc8e4776f5b]) successfully deleted
I20260807 15:01:56.719573 4035 ts_tablet_manager.cc:1918] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:01:56.719650 4035 log.cc:1196] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/0175e11870524ec78754dbf082fbc6e0
I20260807 15:01:56.719898 4035 ts_tablet_manager.cc:1939] T 0175e11870524ec78754dbf082fbc6e0 P 5745462d6b01477f8418f5f872afbefd: Deleting consensus metadata
I20260807 15:01:56.720361 3343 catalog_manager.cc:4841] TS 5745462d6b01477f8418f5f872afbefd (127.1.127.194:36535): tablet 0175e11870524ec78754dbf082fbc6e0 (table test-table [id=6a263dab53a94236bbfb6bc8e4776f5b]) successfully deleted
W20260807 15:01:56.722627 3343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:32865 (error 108)
I20260807 15:01:56.722923 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 3580
I20260807 15:01:56.729756 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:56.729458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:32865 (local address 127.0.0.1:57930)
0807 15:01:56.729647 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:56.729710 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":125,"threads_started":1}
W20260807 15:01:56.729969 3343 catalog_manager.cc:4584] TS 222a77be2f6d470db42bfad8e6cb7174 (127.1.127.193:32865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0175e11870524ec78754dbf082fbc6e0: Network error: Client connection negotiation failed: client connection to 127.1.127.193:32865: connect: Connection refused (error 111)
I20260807 15:01:56.730865 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 3730
I20260807 15:01:56.738945 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 3323
2026-08-07T15:01:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7083 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:56 dist-test-slave-8tdl krb5kdc[4040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:56 dist-test-slave-8tdl krb5kdc[4040](info): set up 2 sockets
Aug 07 15:01:56 dist-test-slave-8tdl krb5kdc[4040](info): commencing operation
krb5kdc: starting...
W20260807 15:01:57.830771 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:57 dist-test-slave-8tdl krb5kdc[4040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:57.975446 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33343
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:58.064447 4053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:58.064744 4053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:58.064790 4053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:58.066062 4053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:01:58.066143 4053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:58.066167 4053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:58.066190 4053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:01:58.066219 4053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:01:58.067845 4053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:58.068177 4053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:58.068432 4053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:58.073877 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.073951 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.073877 4059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:58.074342 4053 server_base.cc:1034] running on GCE node
I20260807 15:01:58.074636 4053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:58.075027 4053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:58.076256 4053 hybrid_clock.cc:648] HybridClock initialized: now 1786114918076246 us; error 46 us; skew 500 ppm
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:58.079604 4053 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:01:58.080190 4053 webserver.cc:460] Webserver started at http://127.1.127.254:42331/ using document root <none> and password file <none>
I20260807 15:01:58.080509 4053 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:58.080593 4053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:58.080757 4053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:58.081965 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "2f0c4375eee34075bdace0ff600e65d7"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "00da4721d52a215b7c7ec5b14d1f8662"
server_key_iv: "326dba4fda0a3610d24837ab8ae0eeb5"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.082381 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "2f0c4375eee34075bdace0ff600e65d7"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "00da4721d52a215b7c7ec5b14d1f8662"
server_key_iv: "326dba4fda0a3610d24837ab8ae0eeb5"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.084079 4053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:01:58.085079 4068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.085327 4053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:58.085451 4053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "2f0c4375eee34075bdace0ff600e65d7"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "00da4721d52a215b7c7ec5b14d1f8662"
server_key_iv: "326dba4fda0a3610d24837ab8ae0eeb5"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.085551 4053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:58.096223 4053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:58.098569 4053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:58.098795 4053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:58.103152 4053 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:33343
I20260807 15:01:58.103216 4124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:33343 every 8 connection(s)
I20260807 15:01:58.103615 4053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:01:58.104574 4125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:01:58.107304 4125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7: Bootstrap starting.
I20260807 15:01:58.107991 4125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:01:58.108319 4125 log.cc:824] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:01:58.109253 4125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7: No bootstrap required, opened a new log
I20260807 15:01:58.111280 4125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } }
I20260807 15:01:58.111469 4125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:01:58.111498 4125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0c4375eee34075bdace0ff600e65d7, State: Initialized, Role: FOLLOWER
I20260807 15:01:58.111673 4125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } }
I20260807 15:01:58.111781 4125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:01:58.111809 4125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:01:58.111845 4125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:01:58.112061 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4053
I20260807 15:01:58.112162 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:01:58.112334 1535 external_mini_cluster.cc:1428] Setting key 2af06d0bff000b715654ef9b6735ac48
I20260807 15:01:58.112715 4125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } }
I20260807 15:01:58.112871 4125 leader_election.cc:302] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f0c4375eee34075bdace0ff600e65d7; no voters:
I20260807 15:01:58.113197 4125 leader_election.cc:288] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:01:58.113432 4129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:01:58.113606 4129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2f0c4375eee34075bdace0ff600e65d7, State: Running, Role: LEADER
I20260807 15:01:58.113729 4125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:01:58.113741 4129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } }
I20260807 15:01:58.114337 4131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0c4375eee34075bdace0ff600e65d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } } }
I20260807 15:01:58.114447 4131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:58.114354 4132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0c4375eee34075bdace0ff600e65d7. Latest consensus state: current_term: 1 leader_uuid: "2f0c4375eee34075bdace0ff600e65d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0c4375eee34075bdace0ff600e65d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33343 } } }
I20260807 15:01:58.114497 4132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:01:58.114754 4139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:01:58.115420 4139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:01:58.117100 4147 mvcc.cc:205] Tried to move back new op lower bound from 7315926704605356032 to 7315926704594993152. Current Snapshot: MvccSnapshot[applied={T|T < 7315926704605356032}]
I20260807 15:01:58.117565 4139 catalog_manager.cc:1347] Generated new cluster ID: 4001a41371014c219ccf2c71e34af4fe
I20260807 15:01:58.117617 4139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:58.127156 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.112982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.1.127.254:33343)
0807 15:01:58.113327 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:58.113337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:58.113389 (+ 52us) server_negotiation.cc:407] Connection header received
0807 15:01:58.113534 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:58.113543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:58.113615 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:58.114766 (+ 1151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:58.114952 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.116854 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.118023 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.118742 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.118869 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.122049 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:58.122055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:58.122056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:58.122072 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:58.124954 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.125506 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.125510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.125510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.125666 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.125927 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.125928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.125928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.126322 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:58.126504 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:58.126633 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 15:01:58.126857 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":219,"threads_started":1}
I20260807 15:01:58.128894 4139 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:01:58.129537 4139 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:01:58.140625 4139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f0c4375eee34075bdace0ff600e65d7: Generated new TSK 0
I20260807 15:01:58.140919 4139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:01:58.180024 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:58.286111 4151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:58.286367 4151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:58.286396 4151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:58.287882 4151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:58.287954 4151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:58.287986 4151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:01:58.290030 4151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:58.290292 4151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:58.290508 4151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:58.296321 4160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.296325 4158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.296375 4157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:58.296734 4151 server_base.cc:1034] running on GCE node
I20260807 15:01:58.296991 4151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:58.297318 4151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:58.298511 4151 hybrid_clock.cc:648] HybridClock initialized: now 1786114918298480 us; error 51 us; skew 500 ppm
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:58.301783 4151 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:01:58.302394 4151 webserver.cc:460] Webserver started at http://127.1.127.193:34065/ using document root <none> and password file <none>
I20260807 15:01:58.302660 4151 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:58.302794 4151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:58.302982 4151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:58.304210 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "ff51f4fe404a489ebbcb2f80f5983fc8"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "8ae23c6bbced4dedb2c4ccbe9d4bf836"
server_key_iv: "49657b98c93d5c7e3466c38d5c3ba2fc"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.304636 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "ff51f4fe404a489ebbcb2f80f5983fc8"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "8ae23c6bbced4dedb2c4ccbe9d4bf836"
server_key_iv: "49657b98c93d5c7e3466c38d5c3ba2fc"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.306283 4151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:58.307210 4168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.307405 4151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:01:58.307528 4151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "ff51f4fe404a489ebbcb2f80f5983fc8"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "8ae23c6bbced4dedb2c4ccbe9d4bf836"
server_key_iv: "49657b98c93d5c7e3466c38d5c3ba2fc"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.307615 4151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:58.327807 4151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:58.330010 4151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:58.330209 4151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:58.330472 4151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:58.330868 4151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:58.330927 4151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.330962 4151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:58.331012 4151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.337234 4151 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:34301
I20260807 15:01:58.337286 4285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:34301 every 8 connection(s)
I20260807 15:01:58.337626 4151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:58.347904 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4151
I20260807 15:01:58.348013 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:01:58.348233 1535 external_mini_cluster.cc:1428] Setting key a0c8164196c767c798eee694b761d21c
I20260807 15:01:58.348387 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.338446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37809 (local address 127.1.127.254:33343)
0807 15:01:58.338516 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:58.338519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:58.338657 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:01:58.339268 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:58.339269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:58.339294 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:58.339381 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:58.340487 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.341028 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.342276 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.342486 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.342560 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.345118 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:58.345119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:58.345120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:58.345131 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:58.346651 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.347257 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.347257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.347258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.347316 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.347599 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.347600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.347600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.347718 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:58.347773 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:58.348177 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 15:01:58.348259 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:01:58.348688 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.338350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33343 (local address 127.1.127.193:37809)
0807 15:01:58.338536 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:58.338582 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:58.339139 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:58.339515 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:58.339519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:58.340144 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:58.340349 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.340354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.341149 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.341150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.342024 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.342027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.342584 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.342585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.342774 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.343459 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:58.343468 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:58.345016 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:58.346895 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.346904 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.346906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.347173 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.347421 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.347421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.347422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.347537 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.347793 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:58.347800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:58.347998 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 15:01:58.348145 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 15:01:58.348351 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260807 15:01:58.349339 4286 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33343
I20260807 15:01:58.349459 4286 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:58.349655 4286 heartbeater.cc:511] Master 127.1.127.254:33343 requested a full tablet report, sending...
I20260807 15:01:58.350242 4087 ts_manager.cc:194] Registered new tserver with Master: ff51f4fe404a489ebbcb2f80f5983fc8 (127.1.127.193:34301)
I20260807 15:01:58.350929 4087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:37809
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:01:58.383734 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:58.472960 4292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:58.473196 4292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:58.473223 4292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:58.474403 4292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:58.474442 4292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:58.474471 4292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:01:58.476065 4292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:58.476287 4292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:58.476476 4292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:58.482174 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.482245 4298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:58.482306 4292 server_base.cc:1034] running on GCE node
W20260807 15:01:58.482187 4300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:58.482803 4292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:58.483234 4292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:58.484437 4292 hybrid_clock.cc:648] HybridClock initialized: now 1786114918484420 us; error 31 us; skew 500 ppm
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:58.487557 4292 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:01:58.488023 4292 webserver.cc:460] Webserver started at http://127.1.127.194:44363/ using document root <none> and password file <none>
I20260807 15:01:58.488289 4292 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:58.488343 4292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:58.488458 4292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:58.489758 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "8a153f97061e47b8a2f8d153de0fec21"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "fd8279eeb91fabfca481f4d025530c95"
server_key_iv: "4518d9c0ab3e121d1c8c001831d2c714"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.490178 4292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "8a153f97061e47b8a2f8d153de0fec21"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "fd8279eeb91fabfca481f4d025530c95"
server_key_iv: "4518d9c0ab3e121d1c8c001831d2c714"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.491775 4292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:01:58.492808 4308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.493002 4292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:58.493099 4292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "8a153f97061e47b8a2f8d153de0fec21"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "fd8279eeb91fabfca481f4d025530c95"
server_key_iv: "4518d9c0ab3e121d1c8c001831d2c714"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.493163 4292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:58.509014 4292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:58.511747 4292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:58.511961 4292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:58.512193 4292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:58.512553 4292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:58.512598 4292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.512622 4292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:58.512635 4292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.519362 4292 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:44313
I20260807 15:01:58.519508 4426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:44313 every 8 connection(s)
I20260807 15:01:58.519840 4292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:01:58.520490 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4292
I20260807 15:01:58.520624 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:01:58.520843 1535 external_mini_cluster.cc:1428] Setting key d7a853c4933581d68eabdefa0f7926bf
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:01:58.530977 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.520952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:49673 (local address 127.1.127.254:33343)
0807 15:01:58.521023 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:58.521026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:58.521505 (+ 479us) server_negotiation.cc:407] Connection header received
0807 15:01:58.522146 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:58.522147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:58.522159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:58.522207 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:58.523365 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.523888 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.524947 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.525169 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.525252 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.527852 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:58.527854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:58.527854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:58.527863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:58.529371 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.529887 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.529889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.529889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.529953 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.530233 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.530234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.530234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.530358 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:58.530436 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:58.530780 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 15:01:58.530876 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:01:58.531198 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.520840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33343 (local address 127.1.127.194:49673)
0807 15:01:58.521295 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:58.521402 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:58.522029 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:58.522285 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:58.522290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:58.523017 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:58.523237 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.523243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.524010 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.524011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.524848 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.524851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.525265 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.525267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.525409 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.525989 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:58.525998 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:58.527739 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:58.529481 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.529490 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.529492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.529808 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.530028 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.530029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.530029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.530156 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.530426 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:58.530433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:58.530596 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 15:01:58.530752 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 15:01:58.530938 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":355,"threads_started":1}
I20260807 15:01:58.531759 4427 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33343
I20260807 15:01:58.531860 4427 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:58.532018 4427 heartbeater.cc:511] Master 127.1.127.254:33343 requested a full tablet report, sending...
I20260807 15:01:58.532394 4087 ts_manager.cc:194] Registered new tserver with Master: 8a153f97061e47b8a2f8d153de0fec21 (127.1.127.194:44313)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
I20260807 15:01:58.533031 4087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:49673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:01:58.551461 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:01:58.641157 4433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:01:58.641505 4433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:01:58.641537 4433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:01:58.643289 4433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:01:58.643383 4433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:01:58.643433 4433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:01:58.645215 4433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:33343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:01:58.645529 4433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:01:58.645824 4433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:01:58.651530 4441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.651536 4439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:01:58.651665 4438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:01:58.651949 4433 server_base.cc:1034] running on GCE node
I20260807 15:01:58.652189 4433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:01:58.652441 4433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:01:58.653689 4433 hybrid_clock.cc:648] HybridClock initialized: now 1786114918653670 us; error 36 us; skew 500 ppm
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:01:58.662144 4433 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:01:58.662830 4433 webserver.cc:460] Webserver started at http://127.1.127.195:40725/ using document root <none> and password file <none>
I20260807 15:01:58.663122 4433 fs_manager.cc:359] Metadata directory not provided
I20260807 15:01:58.663204 4433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:01:58.663380 4433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:01:58.664569 4433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "8d7498a949e14e4892e1539a91cef037"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "0f2b4ed48bf80d4d3626cd84267a9a07"
server_key_iv: "e523da92119c37bea891c7fe83b860db"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.664999 4433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "8d7498a949e14e4892e1539a91cef037"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "0f2b4ed48bf80d4d3626cd84267a9a07"
server_key_iv: "e523da92119c37bea891c7fe83b860db"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.666606 4433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:01:58.667550 4448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.667790 4433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:01:58.667937 4433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "8d7498a949e14e4892e1539a91cef037"
format_stamp: "Formatted at 2026-08-07 15:01:58 on dist-test-slave-8tdl"
server_key: "0f2b4ed48bf80d4d3626cd84267a9a07"
server_key_iv: "e523da92119c37bea891c7fe83b860db"
server_key_version: "encryptionkey@0"
I20260807 15:01:58.668020 4433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:01:58.680172 4433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:01:58.682390 4433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:01:58.682518 4433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:01:58.682819 4433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:01:58.683176 4433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:01:58.683236 4433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.683280 4433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:01:58.683305 4433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:01:58.689579 4433 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:36399
I20260807 15:01:58.689663 4566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:36399 every 8 connection(s)
I20260807 15:01:58.690029 4433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:01:58.698531 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4433
I20260807 15:01:58.698637 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:01:58.698945 1535 external_mini_cluster.cc:1428] Setting key 250164fea1d227671c0ce7ae0c50b02d
I20260807 15:01:58.701220 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.690948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:39563 (local address 127.1.127.254:33343)
0807 15:01:58.690997 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:58.690999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:58.691291 (+ 292us) server_negotiation.cc:407] Connection header received
0807 15:01:58.691846 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:58.691847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:58.691862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:58.691922 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:58.692963 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.693482 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.694520 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.694836 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.694926 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.697522 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:01:58.697524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:01:58.697524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:01:58.697532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:01:58.699437 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.700005 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.700007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.700007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.700080 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:01:58.700498 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:01:58.700499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:01:58.700499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:01:58.700622 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:01:58.700679 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:01:58.701069 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:01:58.701151 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:01:58.701383 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:01:58.690928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:33343 (local address 127.1.127.195:39563)
0807 15:01:58.691154 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 15:01:58.691195 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:01:58.691725 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:01:58.691997 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:01:58.692002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:01:58.692625 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:01:58.692832 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.692837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.693593 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.693594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.694413 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:01:58.694415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:01:58.694944 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:01:58.694945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:01:58.695085 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:01:58.695606 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:01:58.695614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:01:58.697416 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:01:58.699517 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.699527 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.699530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.699906 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.700180 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:01:58.700181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:01:58.700181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:01:58.700406 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:01:58.700704 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:01:58.700710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:01:58.700845 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 15:01:58.701008 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 15:01:58.701152 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":135,"threads_started":1}
I20260807 15:01:58.701998 4567 heartbeater.cc:347] Connected to a master server at 127.1.127.254:33343
I20260807 15:01:58.702147 4567 heartbeater.cc:464] Registering TS with master...
I20260807 15:01:58.702366 4567 heartbeater.cc:511] Master 127.1.127.254:33343 requested a full tablet report, sending...
I20260807 15:01:58.702929 4087 ts_manager.cc:194] Registered new tserver with Master: 8d7498a949e14e4892e1539a91cef037 (127.1.127.195:36399)
I20260807 15:01:58.703343 4087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:39563
I20260807 15:01:58.711844 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:01:58.726105 4130 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:01:58.724976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.1.127.254:33343)
0807 15:01:58.725052 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:01:58.725054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:01:58.725212 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:01:58.725296 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:01:58.725297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:01:58.725311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:01:58.725381 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:01:58.726004 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34318: BlockingRecv error: recv got EOF from 127.0.0.1:34318 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:01:58.727285 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4151
I20260807 15:01:58.738814 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4292
I20260807 15:01:58.745132 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4433
I20260807 15:01:58.751291 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4053
2026-08-07T15:01:58Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2011 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4579](info): set up 2 sockets
Aug 07 15:01:58 dist-test-slave-8tdl krb5kdc[4579](info): commencing operation
krb5kdc: starting...
W20260807 15:01:59.854840 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:01:59 dist-test-slave-8tdl krb5kdc[4579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:01:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:01:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:01:59.994836 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41361
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:00.085075 4592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:00.085320 4592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:00.085354 4592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:00.086545 4592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:00.086585 4592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:00.086601 4592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:00.086615 4592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:00.086629 4592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:00.088302 4592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:00.088531 4592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:00.088717 4592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:00.093842 4597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:00.093827 4598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:00.093828 4601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:00.094102 4592 server_base.cc:1034] running on GCE node
I20260807 15:02:00.094564 4592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:00.095077 4592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:00.096283 4592 hybrid_clock.cc:648] HybridClock initialized: now 1786114920096243 us; error 76 us; skew 500 ppm
Aug 07 15:02:00 dist-test-slave-8tdl krb5kdc[4579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114920, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:00.099212 4592 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:00.099773 4592 webserver.cc:460] Webserver started at http://127.1.127.254:41125/ using document root <none> and password file <none>
I20260807 15:02:00.100016 4592 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:00.100088 4592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:00.100247 4592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:00.101450 4592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "03cf64c2ae4846b1880508f7678f9382"
format_stamp: "Formatted at 2026-08-07 15:02:00 on dist-test-slave-8tdl"
server_key: "df09ce4ce4bbae587751a79302e6f8d7"
server_key_iv: "44fc6a1d241080bfa2a59c4d1daad184"
server_key_version: "encryptionkey@0"
I20260807 15:02:00.101838 4592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "03cf64c2ae4846b1880508f7678f9382"
format_stamp: "Formatted at 2026-08-07 15:02:00 on dist-test-slave-8tdl"
server_key: "df09ce4ce4bbae587751a79302e6f8d7"
server_key_iv: "44fc6a1d241080bfa2a59c4d1daad184"
server_key_version: "encryptionkey@0"
I20260807 15:02:00.103426 4592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:00.104419 4610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:00.104689 4592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:00.104816 4592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "03cf64c2ae4846b1880508f7678f9382"
format_stamp: "Formatted at 2026-08-07 15:02:00 on dist-test-slave-8tdl"
server_key: "df09ce4ce4bbae587751a79302e6f8d7"
server_key_iv: "44fc6a1d241080bfa2a59c4d1daad184"
server_key_version: "encryptionkey@0"
I20260807 15:02:00.104910 4592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:00.118314 4592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:00.120505 4592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:00.120719 4592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:00.125289 4592 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41361
I20260807 15:02:00.125352 4666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41361 every 8 connection(s)
I20260807 15:02:00.125765 4592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:00.126847 4667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:00.129422 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382: Bootstrap starting.
I20260807 15:02:00.130082 4667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:00.130414 4667 log.cc:824] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:00.131069 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4592
I20260807 15:02:00.131172 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:00.131388 1535 external_mini_cluster.cc:1428] Setting key f523e466ce9184725d7b8db928ccd2fd
I20260807 15:02:00.131608 4667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382: No bootstrap required, opened a new log
I20260807 15:02:00.133997 4667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } }
I20260807 15:02:00.134191 4667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:00.134243 4667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03cf64c2ae4846b1880508f7678f9382, State: Initialized, Role: FOLLOWER
I20260807 15:02:00.134414 4667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } }
I20260807 15:02:00.134517 4667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:00.134563 4667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:00.134617 4667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:00.135304 4667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } }
I20260807 15:02:00.135443 4667 leader_election.cc:302] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03cf64c2ae4846b1880508f7678f9382; no voters:
I20260807 15:02:00.135653 4667 leader_election.cc:288] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:00.135689 4672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:00.135813 4672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [term 1 LEADER]: Becoming Leader. State: Replica: 03cf64c2ae4846b1880508f7678f9382, State: Running, Role: LEADER
I20260807 15:02:00.135905 4672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } }
I20260807 15:02:00.135974 4667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:00.136276 4673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03cf64c2ae4846b1880508f7678f9382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } } }
I20260807 15:02:00.136361 4673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:00.136612 4674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03cf64c2ae4846b1880508f7678f9382. Latest consensus state: current_term: 1 leader_uuid: "03cf64c2ae4846b1880508f7678f9382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03cf64c2ae4846b1880508f7678f9382" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41361 } } }
I20260807 15:02:00.136718 4674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:00.137073 4677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:00.137679 4677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:00.139420 4677 catalog_manager.cc:1347] Generated new cluster ID: 03e3bd95e99244139b78c5db775caceb
I20260807 15:02:00.139479 4677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:02:00 dist-test-slave-8tdl krb5kdc[4579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:00.145620 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:00.131936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.1.127.254:41361)
0807 15:02:00.132148 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:00.132158 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:00.132197 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:02:00.134044 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:00.134053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:00.134128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:00.134336 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:00.135253 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:00.136709 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:00.137877 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:00.138580 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:00.139082 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:00.141785 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:00.141790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:00.141790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:00.141804 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:00.143856 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:00.144309 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:00.144313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:00.144313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:00.144444 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:00.144677 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:00.144678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:00.144678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:00.144984 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:00.145147 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:00.145266 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:02:00.145401 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260807 15:02:00.146833 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:02:00.166621 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:00.161214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.1.127.254:41361)
0807 15:02:00.161268 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:00.161270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:00.161474 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:02:00.161587 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:00.161589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:00.161604 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:00.161692 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:00.162226 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:00.163035 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:00.164214 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:00.164562 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:00.164644 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:00.165528 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:00.165529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:00.165530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:00.166163 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:00.166188 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:00.166197 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:00.166265 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:00.166487 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 15:02:00.166527 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:02:00.170253 4677 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:00.171025 4677 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:00.173471 4677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03cf64c2ae4846b1880508f7678f9382: Generated new TSK 0
I20260807 15:02:00.173689 4677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:00.183876 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4592
2026-08-07T15:02:00Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1430 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:00 dist-test-slave-8tdl krb5kdc[4701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:00 dist-test-slave-8tdl krb5kdc[4701](info): set up 2 sockets
Aug 07 15:02:00 dist-test-slave-8tdl krb5kdc[4701](info): commencing operation
krb5kdc: starting...
W20260807 15:02:01.319141 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:01.463225 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46111
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:46111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:01.552376 4714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:01.552661 4714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:01.552707 4714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:01.553999 4714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:01.554081 4714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:01.554104 4714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:01.554126 4714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:01.554157 4714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:01.555823 4714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:46111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:01.556154 4714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:01.556420 4714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:01.561688 4723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:01.561700 4721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:01.561713 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:01.562081 4714 server_base.cc:1034] running on GCE node
I20260807 15:02:01.562350 4714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:01.562711 4714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:01.563891 4714 hybrid_clock.cc:648] HybridClock initialized: now 1786114921563875 us; error 35 us; skew 500 ppm
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:01.566807 4714 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:01.567433 4714 webserver.cc:460] Webserver started at http://127.1.127.254:33405/ using document root <none> and password file <none>
I20260807 15:02:01.567740 4714 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:01.567816 4714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:01.567974 4714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:01.569087 4714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "6be27230a1b84a299879be776b31996c"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "384882f22b44fc5871134e2382dcdc62"
server_key_iv: "7eb8c57eff89df86fc733147a8b478d2"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.569483 4714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "6be27230a1b84a299879be776b31996c"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "384882f22b44fc5871134e2382dcdc62"
server_key_iv: "7eb8c57eff89df86fc733147a8b478d2"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.571026 4714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:01.571787 4730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:01.571970 4714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:02:01.572082 4714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "6be27230a1b84a299879be776b31996c"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "384882f22b44fc5871134e2382dcdc62"
server_key_iv: "7eb8c57eff89df86fc733147a8b478d2"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.572160 4714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:01.594556 4714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:01.596944 4714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:01.597149 4714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:01.601779 4714 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:46111
I20260807 15:02:01.601822 4783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:46111 every 8 connection(s)
I20260807 15:02:01.602252 4714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:01.603701 4784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:01.606878 4784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c: Bootstrap starting.
I20260807 15:02:01.607515 4784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:01.607780 4784 log.cc:824] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:01.608642 4784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c: No bootstrap required, opened a new log
I20260807 15:02:01.610106 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4714
I20260807 15:02:01.610239 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:01.610435 1535 external_mini_cluster.cc:1428] Setting key 1262a8d8016ed6725b396409a8f6f648
I20260807 15:02:01.610596 4784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } }
I20260807 15:02:01.610867 4784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:01.610929 4784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be27230a1b84a299879be776b31996c, State: Initialized, Role: FOLLOWER
I20260807 15:02:01.611094 4784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } }
I20260807 15:02:01.611207 4784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:01.611255 4784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:01.611320 4784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:01.612839 4784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } }
I20260807 15:02:01.613004 4784 leader_election.cc:302] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6be27230a1b84a299879be776b31996c; no voters:
I20260807 15:02:01.613214 4784 leader_election.cc:288] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:01.613274 4792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:01.613445 4792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [term 1 LEADER]: Becoming Leader. State: Replica: 6be27230a1b84a299879be776b31996c, State: Running, Role: LEADER
I20260807 15:02:01.613543 4784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:01.613551 4792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } }
I20260807 15:02:01.614259 4793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6be27230a1b84a299879be776b31996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } } }
I20260807 15:02:01.614255 4794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6be27230a1b84a299879be776b31996c. Latest consensus state: current_term: 1 leader_uuid: "6be27230a1b84a299879be776b31996c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be27230a1b84a299879be776b31996c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46111 } } }
I20260807 15:02:01.614398 4793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:01.614416 4794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:01.615311 4801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:01.615859 4801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:01.617780 4801 catalog_manager.cc:1347] Generated new cluster ID: 32a9b02f274342dd86c2c456b3fe99ea
I20260807 15:02:01.617842 4801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:01.623835 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:01.611157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.1.127.254:46111)
0807 15:02:01.611404 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:01.611450 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:01.611486 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:02:01.611607 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:01.611613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:01.611658 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:01.611772 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:01.612832 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.614275 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.615490 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.616104 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.616225 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:01.619461 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:01.619467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:01.619467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:01.619483 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:01.621796 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:01.622250 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:01.622254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:01.622254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:01.622410 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:01.622635 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:01.622636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:01.622636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:01.623073 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:01.623304 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:01.623387 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:02:01.623564 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":124,"threads_started":1}
I20260807 15:02:01.634244 4801 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:01.635289 4801 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:01.640712 4801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6be27230a1b84a299879be776b31996c: Generated new TSK 0
I20260807 15:02:01.640975 4801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:01.679889 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:01.772583 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:01.772828 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:01.772856 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:01.774129 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:01.774180 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:01.774209 4814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:01.776062 4814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:01.776345 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:01.776654 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:01.782411 4814 server_base.cc:1034] running on GCE node
W20260807 15:02:01.782413 4821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:01.782574 4823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:01.782447 4820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:01.783022 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:01.783306 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:01.784482 4814 hybrid_clock.cc:648] HybridClock initialized: now 1786114921784452 us; error 42 us; skew 500 ppm
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:01.788242 4814 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:01.788763 4814 webserver.cc:460] Webserver started at http://127.1.127.193:43431/ using document root <none> and password file <none>
I20260807 15:02:01.788961 4814 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:01.789016 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:01.789113 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:01.790284 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "2b5999e0af6d436fb94243d61eb461bb"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "2182e117f8118c8f9d4e15fc2c7b6c24"
server_key_iv: "d649187dca8e52508e9e6f84a6a8734f"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.790921 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "2b5999e0af6d436fb94243d61eb461bb"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "2182e117f8118c8f9d4e15fc2c7b6c24"
server_key_iv: "d649187dca8e52508e9e6f84a6a8734f"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.793113 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:02:01.794230 4830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:01.794482 4814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:01.794595 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "2b5999e0af6d436fb94243d61eb461bb"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "2182e117f8118c8f9d4e15fc2c7b6c24"
server_key_iv: "d649187dca8e52508e9e6f84a6a8734f"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.794667 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:01.808260 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:01.811530 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:01.811657 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:01.811880 4814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:01.812179 4814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:01.812211 4814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:01.812235 4814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:01.812247 4814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:01.818585 4814 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:45861
I20260807 15:02:01.818725 4948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:45861 every 8 connection(s)
I20260807 15:02:01.819080 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:01.826786 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4814
I20260807 15:02:01.826951 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:01.827200 1535 external_mini_cluster.cc:1428] Setting key 0ba8cb3dd23ba6a5b7643fd60651460e
I20260807 15:02:01.831418 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:01.819960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:54795 (local address 127.1.127.254:46111)
0807 15:02:01.820043 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:01.820045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:01.820308 (+ 263us) server_negotiation.cc:407] Connection header received
0807 15:02:01.820859 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:01.820860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:01.820877 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:01.821003 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:01.822211 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.822881 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.824166 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.824409 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.824481 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:01.827551 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:01.827553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:01.827553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:01.827566 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:01.829361 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:01.830053 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:01.830059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:01.830060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:01.830137 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:01.830526 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:01.830527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:01.830527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:01.830740 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:01.830811 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:01.831201 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:02:01.831310 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:02:01.831688 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:01.819917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46111 (local address 127.1.127.193:54795)
0807 15:02:01.820104 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:01.820150 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:01.820727 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:01.821200 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:01.821205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:01.821845 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:01.822074 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:01.822079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.823025 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.823026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:01.823974 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:01.823977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:01.824507 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:01.824507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:01.824632 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:01.825565 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:01.825578 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:01.827361 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:01.829467 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:01.829479 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:01.829482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:01.829906 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:01.830234 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:01.830235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:01.830236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:01.830435 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:01.830815 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:01.830824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:01.831026 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 15:02:01.831153 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:02:01.831374 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260807 15:02:01.832260 4949 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46111
I20260807 15:02:01.832374 4949 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:01.832608 4949 heartbeater.cc:511] Master 127.1.127.254:46111 requested a full tablet report, sending...
I20260807 15:02:01.833223 4747 ts_manager.cc:194] Registered new tserver with Master: 2b5999e0af6d436fb94243d61eb461bb (127.1.127.193:45861)
I20260807 15:02:01.833916 4747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:54795
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:01.863732 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:01.955381 4954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:01.955673 4954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:01.955701 4954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:01.957129 4954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:01.957190 4954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:01.957222 4954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:01.959124 4954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:01.959419 4954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:01.959708 4954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:01.965978 4960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:01.965982 4959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:01.965982 4954 server_base.cc:1034] running on GCE node
W20260807 15:02:01.966105 4962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:01.966517 4954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:01.966863 4954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:01.968050 4954 hybrid_clock.cc:648] HybridClock initialized: now 1786114921968035 us; error 29 us; skew 500 ppm
Aug 07 15:02:01 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:01.971063 4954 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:01.971551 4954 webserver.cc:460] Webserver started at http://127.1.127.194:38479/ using document root <none> and password file <none>
I20260807 15:02:01.971751 4954 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:01.971794 4954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:01.971892 4954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:01.972971 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "a7dbc57a6a8246cf95636acce64c4cff"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "54f8f6f596dce88312a5941f79b05daa"
server_key_iv: "80eb2f02ae71b458defadd162a11dd49"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.973562 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "a7dbc57a6a8246cf95636acce64c4cff"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "54f8f6f596dce88312a5941f79b05daa"
server_key_iv: "80eb2f02ae71b458defadd162a11dd49"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.975322 4954 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:01.976392 4970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:01.976862 4954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:01.977097 4954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "a7dbc57a6a8246cf95636acce64c4cff"
format_stamp: "Formatted at 2026-08-07 15:02:01 on dist-test-slave-8tdl"
server_key: "54f8f6f596dce88312a5941f79b05daa"
server_key_iv: "80eb2f02ae71b458defadd162a11dd49"
server_key_version: "encryptionkey@0"
I20260807 15:02:01.977205 4954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:01.996734 4954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:01.999782 4954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:01.999948 4954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:02.000274 4954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:02.000638 4954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:02.000676 4954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:02.000701 4954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:02.000715 4954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:02.008663 4954 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41327
I20260807 15:02:02.008790 5086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41327 every 8 connection(s)
I20260807 15:02:02.009186 4954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:02.010808 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 4954
I20260807 15:02:02.010957 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:02.011206 1535 external_mini_cluster.cc:1428] Setting key 7ed2dcdfbcf6c2a9388fbe35539a7780
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114921, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:02.020789 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:02.010310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:35385 (local address 127.1.127.254:46111)
0807 15:02:02.010594 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:02.010598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:02.010780 (+ 182us) server_negotiation.cc:407] Connection header received
0807 15:02:02.011466 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:02.011468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:02.011481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:02.011540 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:02.012661 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.013190 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.014462 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.014726 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.014848 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:02.017623 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:02.017625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:02.017625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:02.017641 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:02.019228 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.019717 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.019719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.019719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.019786 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.020073 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.020074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.020075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.020205 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:02.020281 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:02.020594 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 15:02:02.020690 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260807 15:02:02.021016 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:02.010319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46111 (local address 127.1.127.194:35385)
0807 15:02:02.010589 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:02.010635 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:02.011326 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:02.011625 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:02.011630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:02.012327 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:02.012548 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:02.012553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.013298 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.013299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:02.014348 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:02.014351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.014862 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.014863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:02.015012 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:02.015643 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:02.015652 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:02.017502 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:02.019345 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:02.019356 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:02.019359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:02.019627 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:02.019864 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:02.019865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:02.019865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:02.020002 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:02.020282 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:02.020290 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:02.020449 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 15:02:02.020560 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:02:02.020741 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":182,"threads_started":1}
I20260807 15:02:02.021566 5087 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46111
I20260807 15:02:02.021660 5087 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:02.021878 5087 heartbeater.cc:511] Master 127.1.127.254:46111 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:02:02.022552 4747 ts_manager.cc:194] Registered new tserver with Master: a7dbc57a6a8246cf95636acce64c4cff (127.1.127.194:41327)
I20260807 15:02:02.023008 4747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:35385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:02.045524 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:02.133919 5092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:02.134193 5092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:02.134239 5092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:02.135602 5092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:02.135679 5092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:02.135726 5092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:02.137362 5092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:02.137671 5092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:02.137943 5092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:02.143604 5100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:02.143623 5097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:02.143633 5098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:02.144026 5092 server_base.cc:1034] running on GCE node
I20260807 15:02:02.144264 5092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:02.144568 5092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:02.145772 5092 hybrid_clock.cc:648] HybridClock initialized: now 1786114922145744 us; error 36 us; skew 500 ppm
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114922, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:02.148989 5092 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:02.149569 5092 webserver.cc:460] Webserver started at http://127.1.127.195:37333/ using document root <none> and password file <none>
I20260807 15:02:02.149842 5092 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:02.149924 5092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:02.150084 5092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:02.151268 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "4be020edf5674ae7bb0c79414a1785a6"
format_stamp: "Formatted at 2026-08-07 15:02:02 on dist-test-slave-8tdl"
server_key: "d7a5130703a31539c9915831b29e5835"
server_key_iv: "e0cd3c290056ceb8c6032e7c9dc80e2b"
server_key_version: "encryptionkey@0"
I20260807 15:02:02.151707 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "4be020edf5674ae7bb0c79414a1785a6"
format_stamp: "Formatted at 2026-08-07 15:02:02 on dist-test-slave-8tdl"
server_key: "d7a5130703a31539c9915831b29e5835"
server_key_iv: "e0cd3c290056ceb8c6032e7c9dc80e2b"
server_key_version: "encryptionkey@0"
I20260807 15:02:02.153322 5092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:02:02.154336 5107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:02.154788 5092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:02.154987 5092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "4be020edf5674ae7bb0c79414a1785a6"
format_stamp: "Formatted at 2026-08-07 15:02:02 on dist-test-slave-8tdl"
server_key: "d7a5130703a31539c9915831b29e5835"
server_key_iv: "e0cd3c290056ceb8c6032e7c9dc80e2b"
server_key_version: "encryptionkey@0"
I20260807 15:02:02.155141 5092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:02.172065 5092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:02.174968 5092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:02.175150 5092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:02.175401 5092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:02.175849 5092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:02.175894 5092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:02.175973 5092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:02.175987 5092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:02.182663 5092 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:37029
I20260807 15:02:02.182732 5227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:37029 every 8 connection(s)
I20260807 15:02:02.183084 5092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114922, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:02.192389 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5092
I20260807 15:02:02.192529 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:02.192713 1535 external_mini_cluster.cc:1428] Setting key fd8f392d29893f13e3bb721b98b4721f
I20260807 15:02:02.193889 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:02.183918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46721 (local address 127.1.127.254:46111)
0807 15:02:02.183997 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:02.184000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:02.184270 (+ 270us) server_negotiation.cc:407] Connection header received
0807 15:02:02.184848 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:02.184850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:02.184862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:02.184996 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:02.186120 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.186644 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.187797 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.188063 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.188128 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:02.190727 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:02.190730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:02.190730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:02.190740 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:02.192226 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.192783 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.192784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.192785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.192858 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.193130 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.193131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.193131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.193257 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:02.193332 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:02.193691 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 15:02:02.193782 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:02:02.194105 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:02.183854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46111 (local address 127.1.127.195:46721)
0807 15:02:02.184105 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:02.184149 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:02.184713 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:02.185110 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:02.185114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:02.185752 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:02.185959 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:02.185964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.186810 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.186811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:02.187696 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:02.187699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.188153 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.188153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:02.188285 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:02.188881 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:02.188890 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:02.190557 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:02.192361 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:02.192374 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:02.192377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:02.192701 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:02.192930 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:02.192932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:02.192932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:02.193058 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:02.193334 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:02.193403 (+ 69us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:02.193540 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:02:02.193656 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:02:02.193837 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":109,"threads_started":1}
I20260807 15:02:02.194594 5229 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46111
I20260807 15:02:02.194736 5229 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:02.194991 5229 heartbeater.cc:511] Master 127.1.127.254:46111 requested a full tablet report, sending...
I20260807 15:02:02.195569 4747 ts_manager.cc:194] Registered new tserver with Master: 4be020edf5674ae7bb0c79414a1785a6 (127.1.127.195:37029)
I20260807 15:02:02.195990 4747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:46721
I20260807 15:02:02.204989 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114922, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114922, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:02.235056 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:02.226417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.1.127.254:46111)
0807 15:02:02.226506 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:02.226509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:02.226818 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:02:02.226856 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:02.226857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:02.226875 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:02.226937 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:02.227676 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.228221 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.229013 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:02.229262 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:02.229333 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:02.232024 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:02.232025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:02.232026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:02.232033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:02.233753 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.234111 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.234112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.234112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.234166 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:02.234355 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:02.234356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:02.234356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:02.234521 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:02.234601 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:02.234765 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:02:02.234909 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 15:02:02.235437 4747 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38274
I20260807 15:02:02.236019 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4814
I20260807 15:02:02.241870 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4954
I20260807 15:02:02.248988 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5092
I20260807 15:02:02.254431 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 4714
2026-08-07T15:02:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2071 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[5243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[5243](info): set up 2 sockets
Aug 07 15:02:02 dist-test-slave-8tdl krb5kdc[5243](info): commencing operation
krb5kdc: starting...
W20260807 15:02:03.382580 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:03 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:03.519856 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36087
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:36087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:03.607795 5262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:03.608134 5262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:03.608180 5262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:03.609485 5262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:03.609573 5262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:03.609596 5262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:03.609618 5262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:03.609647 5262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:03.611430 5262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:36087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:03.611750 5262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:03.612010 5262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:03.617537 5262 server_base.cc:1034] running on GCE node
W20260807 15:02:03.617408 5269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:03.617424 5268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:03.617408 5271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:03.618173 5262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:03.618554 5262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:03.619750 5262 hybrid_clock.cc:648] HybridClock initialized: now 1786114923619755 us; error 72 us; skew 500 ppm
Aug 07 15:02:03 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114923, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:03.623603 5262 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:03.624219 5262 webserver.cc:460] Webserver started at http://127.1.127.254:43173/ using document root <none> and password file <none>
I20260807 15:02:03.624495 5262 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:03.624575 5262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:03.624752 5262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:03.626021 5262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "8d4b19333a4b43cda16ebaa33b3d047c"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "04b09f6e8758f18300d187dcad9b78f2"
server_key_iv: "9555d9685db69db68181978cac87215c"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.626443 5262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "8d4b19333a4b43cda16ebaa33b3d047c"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "04b09f6e8758f18300d187dcad9b78f2"
server_key_iv: "9555d9685db69db68181978cac87215c"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.628203 5262 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:02:03.629315 5278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:03.629575 5262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:03.629701 5262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "8d4b19333a4b43cda16ebaa33b3d047c"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "04b09f6e8758f18300d187dcad9b78f2"
server_key_iv: "9555d9685db69db68181978cac87215c"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.629792 5262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:03.638607 5262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:03.641214 5262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:03.641443 5262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:03.646147 5262 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:36087
I20260807 15:02:03.646184 5332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:36087 every 8 connection(s)
I20260807 15:02:03.646884 5262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:03.647646 5333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:03.650295 5333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c: Bootstrap starting.
I20260807 15:02:03.651048 5333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:03.651312 5333 log.cc:824] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:03.652243 5333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c: No bootstrap required, opened a new log
I20260807 15:02:03.653990 5333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } }
I20260807 15:02:03.654146 5333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:03.654170 5333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d4b19333a4b43cda16ebaa33b3d047c, State: Initialized, Role: FOLLOWER
I20260807 15:02:03.654340 5333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } }
I20260807 15:02:03.654438 5333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:03.654464 5333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:03.654501 5333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:03.655629 5333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } }
I20260807 15:02:03.655730 5333 leader_election.cc:302] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d4b19333a4b43cda16ebaa33b3d047c; no voters:
I20260807 15:02:03.655877 5333 leader_election.cc:288] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:03.655978 5336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:03.656142 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5262
I20260807 15:02:03.656178 5336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [term 1 LEADER]: Becoming Leader. State: Replica: 8d4b19333a4b43cda16ebaa33b3d047c, State: Running, Role: LEADER
I20260807 15:02:03.656221 5333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:03.656250 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:03.656261 5336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } }
I20260807 15:02:03.656451 1535 external_mini_cluster.cc:1428] Setting key 2e9ab544ad72dba92afbadf687b152d8
I20260807 15:02:03.656667 5337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } } }
I20260807 15:02:03.656757 5337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:03.657035 5338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d4b19333a4b43cda16ebaa33b3d047c. Latest consensus state: current_term: 1 leader_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4b19333a4b43cda16ebaa33b3d047c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36087 } } }
I20260807 15:02:03.657207 5338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:03.657593 5342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:03.658329 5342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:03.661725 5342 catalog_manager.cc:1347] Generated new cluster ID: 5dc65ad8b0974623ab166ac9ec4aa316
I20260807 15:02:03.661784 5342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:03.675621 5342 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:03.676532 5342 catalog_manager.cc:1504] Loading token signing keys...
Aug 07 15:02:03 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:03.685985 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:03.657176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.1.127.254:36087)
0807 15:02:03.657366 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:03.657377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:03.657483 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:02:03.659144 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:03.659152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:03.659221 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:03.659368 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:03.660627 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.662634 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.668496 (+ 5862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.669399 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.669539 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:03.681275 (+ 11736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:03.681282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:03.681283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:03.681302 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:03.683666 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:03.684149 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:03.684153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:03.684153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:03.684306 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:03.684534 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:03.684535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:03.684535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:03.684864 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:03.685025 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:03.685539 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 15:02:03.685714 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260807 15:02:03.686291 5342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d4b19333a4b43cda16ebaa33b3d047c: Generated new TSK 0
I20260807 15:02:03.686511 5342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:03.728189 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:03.825686 5360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:03.825970 5360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:03.826004 5360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:03.827601 5360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:03.827663 5360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:03.827703 5360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:03.829852 5360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:03.830199 5360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:03.830911 5360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:03.836612 5366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:03.836632 5365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:03.836640 5368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:03.837126 5360 server_base.cc:1034] running on GCE node
I20260807 15:02:03.837369 5360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:03.837602 5360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:03.838766 5360 hybrid_clock.cc:648] HybridClock initialized: now 1786114923838704 us; error 78 us; skew 500 ppm
Aug 07 15:02:03 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114923, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:03.841614 5360 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:03.842160 5360 webserver.cc:460] Webserver started at http://127.1.127.193:34377/ using document root <none> and password file <none>
I20260807 15:02:03.842403 5360 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:03.842465 5360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:03.842659 5360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:03.843832 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "70da8ade72a34fbaa83f786e954a613a"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "092931164e277e4d3361a4ac847f1e1c"
server_key_iv: "90f9ef3a1b0ab4fcfaec73cb38adfe8f"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.844210 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "70da8ade72a34fbaa83f786e954a613a"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "092931164e277e4d3361a4ac847f1e1c"
server_key_iv: "90f9ef3a1b0ab4fcfaec73cb38adfe8f"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.845665 5360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:03.846575 5376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:03.846812 5360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:03.846939 5360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "70da8ade72a34fbaa83f786e954a613a"
format_stamp: "Formatted at 2026-08-07 15:02:03 on dist-test-slave-8tdl"
server_key: "092931164e277e4d3361a4ac847f1e1c"
server_key_iv: "90f9ef3a1b0ab4fcfaec73cb38adfe8f"
server_key_version: "encryptionkey@0"
I20260807 15:02:03.847043 5360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:03.864104 5360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:03.866809 5360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:03.867003 5360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:03.867288 5360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:03.867660 5360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:03.867717 5360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:03.867756 5360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:03.867803 5360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:03.874503 5360 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:41389
I20260807 15:02:03.874569 5491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:41389 every 8 connection(s)
I20260807 15:02:03.875000 5360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:03.876325 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5360
I20260807 15:02:03.876422 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:03.876677 1535 external_mini_cluster.cc:1428] Setting key 23031b3c640d5467194b8e86ae553436
Aug 07 15:02:03 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114923, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:02:03.887703 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:03.875888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:51965 (local address 127.1.127.254:36087)
0807 15:02:03.875988 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:03.875990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:03.876234 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:02:03.877779 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:03.877780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:03.877796 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:03.877861 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:03.878615 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.879266 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.880413 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.880659 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.880750 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:03.883857 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:03.883859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:03.883860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:03.883870 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:03.885824 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:03.886408 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:03.886411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:03.886411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:03.886486 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:03.886911 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:03.886913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:03.886914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:03.887110 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:03.887198 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:03.887533 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 15:02:03.887629 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:02:03.887974 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:03.875887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36087 (local address 127.1.127.193:51965)
0807 15:02:03.876089 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:03.876141 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:03.877635 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:03.877959 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:03.877967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:03.878274 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:03.878499 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:03.878504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.879381 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.879382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:03.880298 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:03.880300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:03.880761 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:03.880762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:03.880920 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:03.881657 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:03.881667 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:03.883724 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:03.885960 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:03.885973 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:03.885976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:03.886303 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:03.886590 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:03.886591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:03.886591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:03.886821 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:03.887209 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:03.887217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:03.887369 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 15:02:03.887513 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 15:02:03.887673 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260807 15:02:03.888832 5492 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36087
I20260807 15:02:03.889002 5492 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:03.889178 5492 heartbeater.cc:511] Master 127.1.127.254:36087 requested a full tablet report, sending...
I20260807 15:02:03.889895 5295 ts_manager.cc:194] Registered new tserver with Master: 70da8ade72a34fbaa83f786e954a613a (127.1.127.193:41389)
I20260807 15:02:03.890630 5295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:51965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:03.911733 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:03.998309 5497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:03.998548 5497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:03.998574 5497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:03.999799 5497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:03.999850 5497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:03.999879 5497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:04.001482 5497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:04.001729 5497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:04.001947 5497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:04.009512 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:04.009586 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:04.009727 5506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:04.009802 5497 server_base.cc:1034] running on GCE node
I20260807 15:02:04.010465 5497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:04.011003 5497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:04.012301 5497 hybrid_clock.cc:648] HybridClock initialized: now 1786114924012238 us; error 74 us; skew 500 ppm
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:04.016078 5497 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:04.016657 5497 webserver.cc:460] Webserver started at http://127.1.127.194:40489/ using document root <none> and password file <none>
I20260807 15:02:04.016937 5497 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:04.017031 5497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:04.017207 5497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:04.018572 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "bb6d30016e7e459998c909896727fece"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "6eed1d9e74670a4a3f972206aee1b4e8"
server_key_iv: "e083a797df23a1971cf2db57206ec3fc"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.019080 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "bb6d30016e7e459998c909896727fece"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "6eed1d9e74670a4a3f972206aee1b4e8"
server_key_iv: "e083a797df23a1971cf2db57206ec3fc"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.020771 5497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:04.021904 5514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.022192 5497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:04.022303 5497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "bb6d30016e7e459998c909896727fece"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "6eed1d9e74670a4a3f972206aee1b4e8"
server_key_iv: "e083a797df23a1971cf2db57206ec3fc"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.022400 5497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:04.036710 5497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:04.039016 5497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:04.039247 5497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:04.039529 5497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:04.039915 5497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:04.039974 5497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.040019 5497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:04.040060 5497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.046383 5497 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41527
I20260807 15:02:04.046428 5633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41527 every 8 connection(s)
I20260807 15:02:04.046844 5497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:04.048180 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5497
I20260807 15:02:04.048271 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:04.048478 1535 external_mini_cluster.cc:1428] Setting key 44c737b45e4d206015bd082c84cb9ec2
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:02:04.059592 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.047670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:38431 (local address 127.1.127.254:36087)
0807 15:02:04.047752 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:04.047755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:04.047968 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:02:04.049407 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:04.049408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:04.049421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:04.049472 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:04.050317 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.051038 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.052829 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.053063 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.053131 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.055951 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:04.055953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:04.055954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:04.055965 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:04.057921 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.058422 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.058424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.058424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.058501 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.058867 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.058868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.058868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.059004 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:04.059087 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:04.059389 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 15:02:04.059481 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:02:04.059818 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.047582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36087 (local address 127.1.127.194:38431)
0807 15:02:04.047807 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:04.047864 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:04.049292 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:04.049562 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:04.049567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:04.049893 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:04.050209 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:04.050215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.051207 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.051208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:04.052717 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:04.052721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.053177 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.053179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:04.053352 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.053951 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:04.053963 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:04.055797 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:04.058015 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:04.058025 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:04.058028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:04.058334 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:04.058599 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:04.058601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:04.058601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:04.058795 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:04.059080 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:04.059088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:04.059245 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 15:02:04.059356 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:02:04.059539 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260807 15:02:04.060566 5634 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36087
I20260807 15:02:04.060712 5634 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:04.060945 5634 heartbeater.cc:511] Master 127.1.127.254:36087 requested a full tablet report, sending...
I20260807 15:02:04.061390 5295 ts_manager.cc:194] Registered new tserver with Master: bb6d30016e7e459998c909896727fece (127.1.127.194:41527)
I20260807 15:02:04.061883 5295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:38431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:04.084118 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:04.169684 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:04.169952 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:04.169996 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:04.171347 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:04.171430 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:04.171475 5639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:04.173216 5639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:04.173534 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:04.173807 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:04.179731 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:04.179698 5645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:04.179698 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:04.180042 5639 server_base.cc:1034] running on GCE node
I20260807 15:02:04.180301 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:04.180576 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:04.181743 5639 hybrid_clock.cc:648] HybridClock initialized: now 1786114924181681 us; error 84 us; skew 500 ppm
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:04.184850 5639 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:04.185426 5639 webserver.cc:460] Webserver started at http://127.1.127.195:44339/ using document root <none> and password file <none>
I20260807 15:02:04.185719 5639 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:04.185799 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:04.185992 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:04.187347 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "6571e47cb73243d9b419f3b5b50636e6"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "7b23116042507d90ce4a465d2bf126bf"
server_key_iv: "5e254d82adc53797047a4c4bc5ff2bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.187790 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "6571e47cb73243d9b419f3b5b50636e6"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "7b23116042507d90ce4a465d2bf126bf"
server_key_iv: "5e254d82adc53797047a4c4bc5ff2bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.189370 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:04.190330 5654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.190605 5639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:04.190776 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6571e47cb73243d9b419f3b5b50636e6"
format_stamp: "Formatted at 2026-08-07 15:02:04 on dist-test-slave-8tdl"
server_key: "7b23116042507d90ce4a465d2bf126bf"
server_key_iv: "5e254d82adc53797047a4c4bc5ff2bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:04.190878 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:04.200407 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:04.202965 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:04.203145 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:04.203400 5639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:04.203754 5639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:04.203814 5639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.203859 5639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:04.203928 5639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:04.210237 5639 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:42991
I20260807 15:02:04.210285 5775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:42991 every 8 connection(s)
I20260807 15:02:04.210659 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:04.220525 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5639
I20260807 15:02:04.220623 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:04.220860 1535 external_mini_cluster.cc:1428] Setting key 51093b4a687a57bae4606c7701db0c95
I20260807 15:02:04.221633 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.211571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:41105 (local address 127.1.127.254:36087)
0807 15:02:04.211639 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:04.211641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:04.211895 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:02:04.212800 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:04.212801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:04.212821 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:04.212949 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:04.213721 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.214311 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.215457 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.215659 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.215729 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.218205 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:04.218207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:04.218207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:04.218223 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:04.219731 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.220294 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.220296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.220297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.220400 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.220868 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.220869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.220870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.221059 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:04.221137 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:04.221433 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:02:04.221526 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:02:04.221899 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.211513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36087 (local address 127.1.127.195:41105)
0807 15:02:04.211686 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:04.211732 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:04.212682 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:04.213044 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:04.213049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:04.213330 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:04.213604 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:04.213609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.214407 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.214408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:04.215343 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:04.215346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.215750 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.215751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:04.215858 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.216454 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:04.216462 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:04.218089 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:04.219925 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:04.219933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:04.219936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:04.220197 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:04.220492 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:04.220493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:04.220493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:04.220785 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:04.221139 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:04.221147 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:04.221291 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:02:04.221407 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:02:04.221631 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260807 15:02:04.222517 5777 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36087
I20260807 15:02:04.222645 5777 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:04.222877 5777 heartbeater.cc:511] Master 127.1.127.254:36087 requested a full tablet report, sending...
I20260807 15:02:04.223362 5295 ts_manager.cc:194] Registered new tserver with Master: 6571e47cb73243d9b419f3b5b50636e6 (127.1.127.195:42991)
I20260807 15:02:04.223783 5295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:41105
I20260807 15:02:04.233198 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:02:04.260792 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.251467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.1.127.193:41389)
0807 15:02:04.251731 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:04.251737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:04.251750 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:02:04.251778 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:04.251780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:04.251795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:04.251870 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:04.252505 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.253364 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.254163 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.254507 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.254572 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.257477 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:04.257487 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:04.257488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:04.257496 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:04.259376 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.259781 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.259784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.259785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.259866 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.260090 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.260092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.260092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.260258 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:04.260560 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:04.260621 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 15:02:04.260706 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":154,"threads_started":1}
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:04.269848 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:04.262335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.1.127.254:36087)
0807 15:02:04.262393 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:04.262395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:04.262411 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:02:04.262469 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:04.262470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:04.262479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:04.262531 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:04.263192 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.263705 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.264458 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:04.264675 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:04.264737 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:04.267208 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:04.267210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:04.267210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:04.267218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:04.268455 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.268890 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.268891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.268891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.268953 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:04.269194 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:04.269195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:04.269195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:04.269364 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:04.269451 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:04.269680 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 15:02:04.269771 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 15:02:04.270219 5295 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49306
I20260807 15:02:04.270836 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5360
I20260807 15:02:04.279635 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5497
I20260807 15:02:04.285689 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5639
I20260807 15:02:04.291612 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5262
2026-08-07T15:02:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2036 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5793](info): set up 2 sockets
Aug 07 15:02:04 dist-test-slave-8tdl krb5kdc[5793](info): commencing operation
krb5kdc: starting...
W20260807 15:02:05.379609 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:05 dist-test-slave-8tdl krb5kdc[5793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:05.522893 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36683
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:05.610870 5806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:05.611229 5806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:05.611310 5806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:05.612640 5806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:05.612696 5806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:05.612712 5806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:05.612725 5806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:05.612738 5806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:05.614497 5806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:05.614832 5806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:05.615043 5806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:05.620433 5812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:05.620479 5811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:05.620438 5814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:05.620756 5806 server_base.cc:1034] running on GCE node
I20260807 15:02:05.621176 5806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:05.621539 5806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:05.622762 5806 hybrid_clock.cc:648] HybridClock initialized: now 1786114925622708 us; error 71 us; skew 500 ppm
Aug 07 15:02:05 dist-test-slave-8tdl krb5kdc[5793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114925, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:05.625725 5806 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:05.626160 5806 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:05.626216 5806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:05.626406 5806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:05.627676 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "2cec0606d4b4488db659789ae313c68d"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "431efb3950c73cb333d4e4bd92bf608d"
server_key_iv: "80e7fe5534d4d7396f00687ee567c851"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.628099 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "2cec0606d4b4488db659789ae313c68d"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "431efb3950c73cb333d4e4bd92bf608d"
server_key_iv: "80e7fe5534d4d7396f00687ee567c851"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.629666 5806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:05.630784 5820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:05.631071 5806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:05.631174 5806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "2cec0606d4b4488db659789ae313c68d"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "431efb3950c73cb333d4e4bd92bf608d"
server_key_iv: "80e7fe5534d4d7396f00687ee567c851"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.631237 5806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:05.645107 5806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:05.647642 5806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:05.647773 5806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:05.652055 5806 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:36683
I20260807 15:02:05.652107 5874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:36683 every 8 connection(s)
I20260807 15:02:05.652416 5806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:05.653610 5875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:05.656342 5875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: Bootstrap starting.
I20260807 15:02:05.657058 5875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:05.657382 5875 log.cc:824] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:05.658289 5875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: No bootstrap required, opened a new log
I20260807 15:02:05.659302 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5806
I20260807 15:02:05.659400 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:05.659633 1535 external_mini_cluster.cc:1428] Setting key 6934d1137aed169919fece97b8954aa7
I20260807 15:02:05.660656 5875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } }
I20260807 15:02:05.660945 5875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:05.661033 5875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cec0606d4b4488db659789ae313c68d, State: Initialized, Role: FOLLOWER
I20260807 15:02:05.661230 5875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } }
I20260807 15:02:05.661334 5875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:05.661398 5875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:05.661481 5875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:05.662354 5875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } }
I20260807 15:02:05.662513 5875 leader_election.cc:302] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cec0606d4b4488db659789ae313c68d; no voters:
I20260807 15:02:05.662778 5875 leader_election.cc:288] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:05.662823 5880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:05.662946 5880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [term 1 LEADER]: Becoming Leader. State: Replica: 2cec0606d4b4488db659789ae313c68d, State: Running, Role: LEADER
I20260807 15:02:05.663108 5880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } }
I20260807 15:02:05.663108 5875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:05.663650 5881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cec0606d4b4488db659789ae313c68d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } } }
I20260807 15:02:05.663767 5881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:05.663667 5882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cec0606d4b4488db659789ae313c68d. Latest consensus state: current_term: 1 leader_uuid: "2cec0606d4b4488db659789ae313c68d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cec0606d4b4488db659789ae313c68d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36683 } } }
I20260807 15:02:05.663970 5882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:05.664472 5885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:05.665277 5885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:05.667205 5885 catalog_manager.cc:1347] Generated new cluster ID: f9e48b1059d149ab9b1aed5cbfda7689
I20260807 15:02:05.667269 5885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:02:05 dist-test-slave-8tdl krb5kdc[5793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:05.673262 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:05.660399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.1.127.254:36683)
0807 15:02:05.660678 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:05.660688 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:05.660721 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:02:05.660831 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:05.660837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:05.660892 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:05.661068 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:05.662338 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.664204 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.665394 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.666101 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.666216 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:05.669223 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:05.669227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:05.669227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:05.669241 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:05.671359 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:05.671889 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:05.671892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:05.671893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:05.672036 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:05.672278 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:05.672279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:05.672279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:05.672607 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:05.672754 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:05.672896 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:02:05.673050 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":147,"threads_started":1}
I20260807 15:02:05.677706 5885 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:05.678514 5885 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:05.692165 5885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: Generated new TSK 0
I20260807 15:02:05.692472 5885 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:05.735643 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:05.831106 5902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:05.831310 5902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:05.831336 5902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:05.832743 5902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:05.832805 5902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:05.832849 5902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:05.834967 5902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:05.835242 5902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:05.835510 5902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:05.840842 5908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:05.841059 5910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:05.840857 5907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:05.841136 5902 server_base.cc:1034] running on GCE node
I20260807 15:02:05.841408 5902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:05.841712 5902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:05.842902 5902 hybrid_clock.cc:648] HybridClock initialized: now 1786114925842850 us; error 77 us; skew 500 ppm
Aug 07 15:02:05 dist-test-slave-8tdl krb5kdc[5793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114925, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:05.845661 5902 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:05.846104 5902 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:05.846180 5902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:05.846352 5902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:05.847618 5902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "e86f671748ad41d39672ef646b88cf4c"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "6a72675f226928bb4f52a6b8c126069a"
server_key_iv: "dc7e9bec8e9d4c12be0961bf4cf4e191"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.847993 5902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "e86f671748ad41d39672ef646b88cf4c"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "6a72675f226928bb4f52a6b8c126069a"
server_key_iv: "dc7e9bec8e9d4c12be0961bf4cf4e191"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.849534 5902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:02:05.850436 5918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:05.850713 5902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:05.850844 5902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "e86f671748ad41d39672ef646b88cf4c"
format_stamp: "Formatted at 2026-08-07 15:02:05 on dist-test-slave-8tdl"
server_key: "6a72675f226928bb4f52a6b8c126069a"
server_key_iv: "dc7e9bec8e9d4c12be0961bf4cf4e191"
server_key_version: "encryptionkey@0"
I20260807 15:02:05.850945 5902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:05.873633 5902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:05.876384 5902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:05.876574 5902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:05.876854 5902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:05.877216 5902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:05.877271 5902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:05.877308 5902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:05.877336 5902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:05.883975 5902 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:37127
I20260807 15:02:05.884274 6037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:37127 every 8 connection(s)
I20260807 15:02:05.884330 5902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:02:05 dist-test-slave-8tdl krb5kdc[5793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114925, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:05.893795 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 5902
I20260807 15:02:05.893932 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:05.894182 1535 external_mini_cluster.cc:1428] Setting key 40584d750843029165788c92eb0c2cb0
I20260807 15:02:05.895313 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:05.885397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:60733 (local address 127.1.127.254:36683)
0807 15:02:05.885527 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:05.885530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:05.885667 (+ 137us) server_negotiation.cc:407] Connection header received
0807 15:02:05.886324 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:05.886326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:05.886339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:05.886391 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:05.887532 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.888055 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.889139 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.889360 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.889457 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:05.892062 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:05.892063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:05.892064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:05.892072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:05.893442 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:05.893953 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:05.893954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:05.893955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:05.894111 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:05.894445 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:05.894447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:05.894448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:05.894612 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:05.894730 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:05.895105 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 15:02:05.895215 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 15:02:05.895476 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:05.885269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36683 (local address 127.1.127.193:60733)
0807 15:02:05.885505 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:05.885562 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:05.886192 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:05.886492 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:05.886496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:05.886804 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:05.887395 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:05.887401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.888182 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.888183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:05.889037 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:05.889040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:05.889476 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:05.889477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:05.889609 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:05.890271 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:05.890280 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:05.891949 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:05.893564 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:05.893572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:05.893575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:05.893821 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:05.894195 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:05.894196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:05.894196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:05.894363 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:05.894713 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:05.894721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:05.894893 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 15:02:05.895011 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:02:05.895219 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260807 15:02:05.896111 6038 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36683
I20260807 15:02:05.896229 6038 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:05.896431 6038 heartbeater.cc:511] Master 127.1.127.254:36683 requested a full tablet report, sending...
I20260807 15:02:05.897068 5838 ts_manager.cc:194] Registered new tserver with Master: e86f671748ad41d39672ef646b88cf4c (127.1.127.193:37127)
I20260807 15:02:05.897716 5838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:60733
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:05.927078 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:06.020625 6044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:06.020884 6044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:06.020912 6044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:06.022194 6044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:06.022248 6044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:06.022279 6044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:06.024021 6044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:06.024305 6044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:06.024569 6044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:06.029860 6050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.029819 6049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.030043 6044 server_base.cc:1034] running on GCE node
W20260807 15:02:06.030048 6052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.030381 6044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:06.030669 6044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:06.031888 6044 hybrid_clock.cc:648] HybridClock initialized: now 1786114926031854 us; error 44 us; skew 500 ppm
Aug 07 15:02:06 dist-test-slave-8tdl krb5kdc[5793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:06.035211 6044 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:06.035631 6044 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:06.035687 6044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:06.035796 6044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:06.036986 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "ccc69f9877214064a4a1c2237904f426"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "560cc1a72a4ae753479f4c06aba3057c"
server_key_iv: "56f911988d10a934781a4125cc510938"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.037366 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "ccc69f9877214064a4a1c2237904f426"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "560cc1a72a4ae753479f4c06aba3057c"
server_key_iv: "56f911988d10a934781a4125cc510938"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.039184 6044 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:06.040199 6059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.040517 6044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:06.040678 6044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "ccc69f9877214064a4a1c2237904f426"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "560cc1a72a4ae753479f4c06aba3057c"
server_key_iv: "56f911988d10a934781a4125cc510938"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.040773 6044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:06.078366 6044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:06.081143 6044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:06.081377 6044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:06.081665 6044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:06.082024 6044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:06.082090 6044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.082129 6044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:06.082178 6044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.088909 6044 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:43543
I20260807 15:02:06.088986 6178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:43543 every 8 connection(s)
I20260807 15:02:06.089263 6044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:06.094180 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6044
I20260807 15:02:06.094302 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:06.094514 1535 external_mini_cluster.cc:1428] Setting key 7c26eb8d0060cd796db5662c81892f56
Aug 07 15:02:06 dist-test-slave-8tdl krb5kdc[5793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:06.100566 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.090177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:39433 (local address 127.1.127.254:36683)
0807 15:02:06.090255 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.090257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.090451 (+ 194us) server_negotiation.cc:407] Connection header received
0807 15:02:06.091082 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.091083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.091095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.091157 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.092398 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.092942 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.094083 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.094406 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.094552 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.097081 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.097082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.097083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:06.097093 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.098952 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.099438 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.099440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.099440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.099512 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.099781 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.099782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.099783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.099951 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.100039 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.100337 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.100454 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:02:06.100670 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.090110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36683 (local address 127.1.127.194:39433)
0807 15:02:06.090317 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.090359 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.090946 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.091327 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.091331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.091651 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:06.092273 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.092280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.093068 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.093069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.093901 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.093903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.094578 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.094579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.094792 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.095464 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:06.095474 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:06.096926 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:06.099065 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:06.099073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:06.099075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:06.099349 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:06.099591 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:06.099592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:06.099592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:06.099708 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:06.100024 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:06.100030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:06.100168 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.100287 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.100434 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260807 15:02:06.101244 6179 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36683
I20260807 15:02:06.101372 6179 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:06.101577 6179 heartbeater.cc:511] Master 127.1.127.254:36683 requested a full tablet report, sending...
I20260807 15:02:06.102054 5838 ts_manager.cc:194] Registered new tserver with Master: ccc69f9877214064a4a1c2237904f426 (127.1.127.194:43543)
I20260807 15:02:06.102613 5838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:39433
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:06.128279 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:06.218396 6184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:06.218637 6184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:06.218729 6184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:06.220103 6184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:06.220157 6184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:06.220189 6184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:06.221880 6184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:06.222170 6184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:06.222438 6184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:06.228209 6193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.228232 6189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.228245 6190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.228579 6184 server_base.cc:1034] running on GCE node
I20260807 15:02:06.228914 6184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:06.229315 6184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:06.230513 6184 hybrid_clock.cc:648] HybridClock initialized: now 1786114926230457 us; error 69 us; skew 500 ppm
Aug 07 15:02:06 dist-test-slave-8tdl krb5kdc[5793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:06.233361 6184 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:06.233774 6184 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:06.233824 6184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:06.233981 6184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:06.235229 6184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "94064edb2625482c930f76eb7f032842"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "91c9de674e7b5699badcf13e28f90b9b"
server_key_iv: "340d9f8eecd163f573347b48b51d1600"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.235718 6184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "94064edb2625482c930f76eb7f032842"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "91c9de674e7b5699badcf13e28f90b9b"
server_key_iv: "340d9f8eecd163f573347b48b51d1600"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.237462 6184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:06.238518 6199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.238896 6184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:06.239039 6184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "94064edb2625482c930f76eb7f032842"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "91c9de674e7b5699badcf13e28f90b9b"
server_key_iv: "340d9f8eecd163f573347b48b51d1600"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.239131 6184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:06.265677 6184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:06.268553 6184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:06.268735 6184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:06.269047 6184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:06.269459 6184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:06.269501 6184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.269526 6184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:06.269539 6184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.276587 6184 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:36013
I20260807 15:02:06.276628 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:36013 every 8 connection(s)
I20260807 15:02:06.276974 6184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:02:06 dist-test-slave-8tdl krb5kdc[5793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114926, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:06.285168 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6184
I20260807 15:02:06.285277 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:06.285501 1535 external_mini_cluster.cc:1428] Setting key bbe3f44d64517cb390f6db1402d321b1
I20260807 15:02:06.288194 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.277947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:53605 (local address 127.1.127.254:36683)
0807 15:02:06.278030 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.278032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.278207 (+ 175us) server_negotiation.cc:407] Connection header received
0807 15:02:06.278786 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.278788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.278800 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.278868 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.280030 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.280675 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.281743 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.281946 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.282014 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.284675 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.284676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.284677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:06.284684 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.286612 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.287187 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.287188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.287188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.287253 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.287554 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.287555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.287555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.287676 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.287735 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.288035 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.288120 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:02:06.288430 6322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.277850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36683 (local address 127.1.127.195:53605)
0807 15:02:06.278035 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.278081 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.278604 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.279055 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.279059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.279342 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:06.279854 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.279859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.280792 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.280792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.281631 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.281634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.282036 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.282036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.282200 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.282902 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:06.282912 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:06.284565 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:06.286740 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:06.286751 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:06.286754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:06.287095 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:06.287348 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:06.287349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:06.287349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:06.287482 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:06.287741 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:06.287747 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:06.287879 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.287972 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.288176 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260807 15:02:06.288951 6320 heartbeater.cc:347] Connected to a master server at 127.1.127.254:36683
I20260807 15:02:06.289062 6320 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:06.289196 6320 heartbeater.cc:511] Master 127.1.127.254:36683 requested a full tablet report, sending...
I20260807 15:02:06.289629 5838 ts_manager.cc:194] Registered new tserver with Master: 94064edb2625482c930f76eb7f032842 (127.1.127.195:36013)
I20260807 15:02:06.290097 5838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:53605
I20260807 15:02:06.298309 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:02:06.307649 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.300096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.1.127.254:36683)
0807 15:02:06.300206 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.300208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.300328 (+ 120us) server_negotiation.cc:407] Connection header received
0807 15:02:06.300443 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.300444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.300457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.300556 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.301252 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.301859 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.302723 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.302981 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.303074 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.304536 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.304537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.304538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:06.304546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.306183 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.306613 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.306614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.306615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.306745 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:06.306989 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:06.306990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:06.306991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:06.307179 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.307307 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.307420 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.307519 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:02:06.309821 5838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:06.311827 5838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:06.320135 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.314986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.1.127.194:43543)
0807 15:02:06.315189 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.315195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.315209 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:02:06.315255 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.315257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.315276 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.315472 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:06.315751 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.317034 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.318765 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.319607 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.319696 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.319760 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.319965 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.320029 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260807 15:02:06.320407 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.314863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:43543 (local address 127.0.0.1:46138)
0807 15:02:06.314997 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.315043 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.315118 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.315495 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.315498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.315501 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:06.315646 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.315650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.317162 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.317163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.318654 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.318657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.319712 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.319713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.319821 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.319842 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.319930 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.320180 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260807 15:02:06.320741 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.315945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:37127 (local address 127.0.0.1:60066)
0807 15:02:06.316098 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.316110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.316169 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.316624 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.316625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.316628 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:06.316684 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.316685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.318089 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.318090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.319115 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.319117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.320471 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.320472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.320545 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.320550 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.320615 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.320660 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260807 15:02:06.321264 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.316235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.1.127.193:37127)
0807 15:02:06.316424 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.316429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.316440 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:06.316483 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.316485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.316502 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.316598 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:06.316756 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.317970 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.319232 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.319856 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.319941 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.320002 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.321087 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.321141 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260807 15:02:06.321352 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.315660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:36013 (local address 127.0.0.1:44172)
0807 15:02:06.315799 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.315810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.316023 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.317444 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.317445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.317448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:06.317488 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.317489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.318640 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.318641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.319924 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.319928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.321095 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.321096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.321162 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.321166 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.321223 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.321267 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":100,"threads_started":1}
I20260807 15:02:06.321616 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.315800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.1.127.195:36013)
0807 15:02:06.317237 (+ 1437us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.317242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.317253 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:06.317294 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.317296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.317323 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.317412 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:06.317560 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.318518 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.320038 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.320900 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.320989 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.321052 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.321461 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.321510 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1396,"thread_start_us":103,"threads_started":1}
I20260807 15:02:06.321566 6107 tablet_service.cc:1467] Processing CreateTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 (DEFAULT_TABLE table=test-table [id=1e31be58621540c9902991edf8ff199c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:06.321978 6107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346e5fbbe08e4f209c0d2276d2d68285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:06.321943 5970 tablet_service.cc:1467] Processing CreateTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 (DEFAULT_TABLE table=test-table [id=1e31be58621540c9902991edf8ff199c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:06.322278 5970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346e5fbbe08e4f209c0d2276d2d68285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:06.323444 6251 tablet_service.cc:1467] Processing CreateTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 (DEFAULT_TABLE table=test-table [id=1e31be58621540c9902991edf8ff199c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:06.323840 6251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346e5fbbe08e4f209c0d2276d2d68285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:06.325362 6338 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Bootstrap starting.
I20260807 15:02:06.326119 6338 tablet_bootstrap.cc:654] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:06.326232 6339 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Bootstrap starting.
I20260807 15:02:06.327077 6339 tablet_bootstrap.cc:654] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:06.327363 6338 log.cc:824] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:06.327854 6339 log.cc:824] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:06.328226 6340 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Bootstrap starting.
I20260807 15:02:06.328724 6339 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: No bootstrap required, opened a new log
I20260807 15:02:06.328787 6339 ts_tablet_manager.cc:1397] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:02:06.330315 6339 raft_consensus.cc:348] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.330464 6339 raft_consensus.cc:374] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:06.330492 6339 raft_consensus.cc:729] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86f671748ad41d39672ef646b88cf4c, State: Initialized, Role: FOLLOWER
I20260807 15:02:06.330655 6339 consensus_queue.cc:260] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.331076 6339 ts_tablet_manager.cc:1428] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:06.331235 6038 heartbeater.cc:503] Master 127.1.127.254:36683 was elected leader, sending a full tablet report...
I20260807 15:02:06.331446 6338 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: No bootstrap required, opened a new log
I20260807 15:02:06.331532 6338 ts_tablet_manager.cc:1397] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:02:06.331856 6340 tablet_bootstrap.cc:654] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:06.332583 6340 log.cc:824] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:06.333461 6340 tablet_bootstrap.cc:492] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: No bootstrap required, opened a new log
I20260807 15:02:06.333580 6340 ts_tablet_manager.cc:1397] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 15:02:06.333993 6338 raft_consensus.cc:348] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.334204 6338 raft_consensus.cc:374] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:06.334290 6338 raft_consensus.cc:729] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc69f9877214064a4a1c2237904f426, State: Initialized, Role: FOLLOWER
I20260807 15:02:06.334429 6338 consensus_queue.cc:260] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.334771 6338 ts_tablet_manager.cc:1428] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:02:06.334915 6179 heartbeater.cc:503] Master 127.1.127.254:36683 was elected leader, sending a full tablet report...
I20260807 15:02:06.335896 6340 raft_consensus.cc:348] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.336110 6340 raft_consensus.cc:374] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:06.336154 6340 raft_consensus.cc:729] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94064edb2625482c930f76eb7f032842, State: Initialized, Role: FOLLOWER
I20260807 15:02:06.336280 6340 consensus_queue.cc:260] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.336581 6320 heartbeater.cc:503] Master 127.1.127.254:36683 was elected leader, sending a full tablet report...
I20260807 15:02:06.336872 6340 ts_tablet_manager.cc:1428] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260807 15:02:06.339874 6180 tablet.cc:2369] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:02:06.385390 6039 tablet.cc:2369] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:06.449390 6343 raft_consensus.cc:482] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:06.449570 6343 raft_consensus.cc:504] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.449980 6343 leader_election.cc:288] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccc69f9877214064a4a1c2237904f426 (127.1.127.194:43543), 94064edb2625482c930f76eb7f032842 (127.1.127.195:36013)
I20260807 15:02:06.454617 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.450318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:45475 (local address 127.1.127.194:43543)
0807 15:02:06.450380 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.450382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.450450 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:02:06.450575 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.450576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.450588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.450661 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:06.450932 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.451742 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.453506 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.454184 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.454276 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.454286 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.454470 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.454521 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:02:06.454720 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.450395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:55079 (local address 127.1.127.195:36013)
0807 15:02:06.450445 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.450450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.450540 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:02:06.450574 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.450575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.450586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.450661 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:06.450905 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.451752 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.453492 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.454187 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.454275 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.454286 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.454576 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.454621 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:02:06.454734 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.450290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:36013 (local address 127.1.127.193:55079)
0807 15:02:06.450432 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.450442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.450517 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.450700 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.450701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.450706 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:06.450808 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.450809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.451910 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.451911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.453376 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.453382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.454295 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.454297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.454384 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.454386 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.454451 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.454515 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":72,"thread_start_us":95,"threads_started":1}
I20260807 15:02:06.455075 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.450194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:43543 (local address 127.1.127.193:45475)
0807 15:02:06.450375 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.450389 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.450484 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.450701 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.450702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.450704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:06.450785 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.450787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.451910 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.451911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.453397 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:06.453399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.454297 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.454297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:06.454378 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.454383 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.454451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.454515 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":84,"thread_start_us":90,"threads_started":1}
I20260807 15:02:06.455367 6272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346e5fbbe08e4f209c0d2276d2d68285" candidate_uuid: "e86f671748ad41d39672ef646b88cf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94064edb2625482c930f76eb7f032842" is_pre_election: true
I20260807 15:02:06.455416 6131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346e5fbbe08e4f209c0d2276d2d68285" candidate_uuid: "e86f671748ad41d39672ef646b88cf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccc69f9877214064a4a1c2237904f426" is_pre_election: true
I20260807 15:02:06.455542 6272 raft_consensus.cc:2454] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86f671748ad41d39672ef646b88cf4c in term 0.
I20260807 15:02:06.455605 6131 raft_consensus.cc:2454] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e86f671748ad41d39672ef646b88cf4c in term 0.
I20260807 15:02:06.455839 5919 leader_election.cc:302] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94064edb2625482c930f76eb7f032842, e86f671748ad41d39672ef646b88cf4c; no voters:
I20260807 15:02:06.456010 6343 raft_consensus.cc:2781] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:06.456122 6343 raft_consensus.cc:482] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:06.456144 6343 raft_consensus.cc:3037] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:06.456862 6343 raft_consensus.cc:504] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.457020 6343 leader_election.cc:288] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [CANDIDATE]: Term 1 election: Requested vote from peers ccc69f9877214064a4a1c2237904f426 (127.1.127.194:43543), 94064edb2625482c930f76eb7f032842 (127.1.127.195:36013)
I20260807 15:02:06.457230 6272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346e5fbbe08e4f209c0d2276d2d68285" candidate_uuid: "e86f671748ad41d39672ef646b88cf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94064edb2625482c930f76eb7f032842"
I20260807 15:02:06.457222 6131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346e5fbbe08e4f209c0d2276d2d68285" candidate_uuid: "e86f671748ad41d39672ef646b88cf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccc69f9877214064a4a1c2237904f426"
I20260807 15:02:06.457327 6272 raft_consensus.cc:3037] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:06.457373 6131 raft_consensus.cc:3037] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:06.458123 6272 raft_consensus.cc:2454] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86f671748ad41d39672ef646b88cf4c in term 1.
I20260807 15:02:06.458272 6131 raft_consensus.cc:2454] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e86f671748ad41d39672ef646b88cf4c in term 1.
I20260807 15:02:06.458387 5919 leader_election.cc:302] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94064edb2625482c930f76eb7f032842, e86f671748ad41d39672ef646b88cf4c; no voters:
I20260807 15:02:06.458508 6343 raft_consensus.cc:2781] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:06.458738 6343 raft_consensus.cc:686] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [term 1 LEADER]: Becoming Leader. State: Replica: e86f671748ad41d39672ef646b88cf4c, State: Running, Role: LEADER
I20260807 15:02:06.458858 6343 consensus_queue.cc:237] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } }
I20260807 15:02:06.459802 5838 catalog_manager.cc:5495] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c reported cstate change: term changed from 0 to 1, leader changed from <none> to e86f671748ad41d39672ef646b88cf4c (127.1.127.193). New cstate: current_term: 1 leader_uuid: "e86f671748ad41d39672ef646b88cf4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e86f671748ad41d39672ef646b88cf4c" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 37127 } health_report { overall_health: HEALTHY } } }
I20260807 15:02:06.465709 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.463048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.1.127.193:37127)
0807 15:02:06.463121 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.463123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.463132 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:06.463170 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.463171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.463190 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.463275 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:06.463428 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.464045 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.464829 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:06.465077 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:06.465168 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:06.465356 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:06.465481 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:06.465572 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.465596 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.465618 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:02:06.467672 6131 raft_consensus.cc:1261] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 1 FOLLOWER]: Refusing update from remote peer e86f671748ad41d39672ef646b88cf4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:06.467672 6272 raft_consensus.cc:1261] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 1 FOLLOWER]: Refusing update from remote peer e86f671748ad41d39672ef646b88cf4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:06.468153 6343 consensus_queue.cc:1035] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "94064edb2625482c930f76eb7f032842" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 36013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:06.468320 6349 consensus_queue.cc:1035] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccc69f9877214064a4a1c2237904f426" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 43543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:06.471065 6352 mvcc.cc:205] Tried to move back new op lower bound from 7315926738808659968 to 7315926738776195072. Current Snapshot: MvccSnapshot[applied={T|T < 7315926738808659968}]
I20260807 15:02:06.473366 5837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:06.473469 5837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:06.474383 5837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cec0606d4b4488db659789ae313c68d: Sending DeleteTablet for 3 replicas of tablet 346e5fbbe08e4f209c0d2276d2d68285
I20260807 15:02:06.474773 5970 tablet_service.cc:1514] Processing DeleteTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:06 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:60066
I20260807 15:02:06.474774 6107 tablet_service.cc:1514] Processing DeleteTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:06 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46138
I20260807 15:02:06.475059 6362 tablet_replica.cc:330] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: stopping tablet replica
I20260807 15:02:06.475114 6251 tablet_service.cc:1514] Processing DeleteTablet for tablet 346e5fbbe08e4f209c0d2276d2d68285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:06 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:44172
I20260807 15:02:06.475195 6362 raft_consensus.cc:2229] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:06.475311 6362 raft_consensus.cc:2258] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:06.475350 6364 tablet_replica.cc:330] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: stopping tablet replica
I20260807 15:02:06.475500 6363 tablet_replica.cc:330] T 346e5fbbe08e4f209c0d2276d2d68285 P e86f671748ad41d39672ef646b88cf4c: stopping tablet replica
I20260807 15:02:06.475623 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5902
I20260807 15:02:06.475656 6364 raft_consensus.cc:2229] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:06.476749 6364 raft_consensus.cc:2258] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:06.477234 6362 ts_tablet_manager.cc:1905] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:06.479310 6362 ts_tablet_manager.cc:1918] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:06.479394 6362 log.cc:1196] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/346e5fbbe08e4f209c0d2276d2d68285
I20260807 15:02:06.479673 6362 ts_tablet_manager.cc:1939] T 346e5fbbe08e4f209c0d2276d2d68285 P ccc69f9877214064a4a1c2237904f426: Deleting consensus metadata
I20260807 15:02:06.480388 5824 catalog_manager.cc:4841] TS ccc69f9877214064a4a1c2237904f426 (127.1.127.194:43543): tablet 346e5fbbe08e4f209c0d2276d2d68285 (table test-table [id=1e31be58621540c9902991edf8ff199c]) successfully deleted
I20260807 15:02:06.483834 6364 ts_tablet_manager.cc:1905] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:06.485621 6364 ts_tablet_manager.cc:1918] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:06.485702 6364 log.cc:1196] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/346e5fbbe08e4f209c0d2276d2d68285
I20260807 15:02:06.485986 6364 ts_tablet_manager.cc:1939] T 346e5fbbe08e4f209c0d2276d2d68285 P 94064edb2625482c930f76eb7f032842: Deleting consensus metadata
I20260807 15:02:06.486528 5821 catalog_manager.cc:4841] TS 94064edb2625482c930f76eb7f032842 (127.1.127.195:36013): tablet 346e5fbbe08e4f209c0d2276d2d68285 (table test-table [id=1e31be58621540c9902991edf8ff199c]) successfully deleted
W20260807 15:02:06.487815 5824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:37127 (error 108)
I20260807 15:02:06.488076 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6044
I20260807 15:02:06.489914 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.489727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:37127 (local address 127.0.0.1:60090)
0807 15:02:06.489801 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.489863 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:02:06.490155 5824 catalog_manager.cc:4584] TS e86f671748ad41d39672ef646b88cf4c (127.1.127.193:37127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 346e5fbbe08e4f209c0d2276d2d68285: Network error: Client connection negotiation failed: client connection to 127.1.127.193:37127: connect: Connection refused (error 111)
I20260807 15:02:06.494756 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6184
I20260807 15:02:06.500599 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 5806
2026-08-07T15:02:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2209 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T15:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:06Z Disabled control of system clock
I20260807 15:02:06.532244 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:45545
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:40321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:45545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:02:06.618383 6369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:06.618623 6369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:06.618646 6369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:06.619859 6369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:06.619907 6369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:06.619921 6369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:06.619935 6369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:06.619946 6369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:06.621316 6369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:45545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:45545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:06.621555 6369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:06.621829 6369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:06.627211 6375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.627269 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.627368 6369 server_base.cc:1034] running on GCE node
W20260807 15:02:06.627233 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.627754 6369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:06.628188 6369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:06.629354 6369 hybrid_clock.cc:648] HybridClock initialized: now 1786114926629345 us; error 35 us; skew 500 ppm
I20260807 15:02:06.630954 6369 webserver.cc:460] Webserver started at http://127.1.127.254:33281/ using document root <none> and password file <none>
I20260807 15:02:06.631274 6369 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:06.631350 6369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:06.631506 6369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:06.632655 6369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "fbf00095b6844884995dad5b9a514212"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "25d3732117db1d5c990e703aa2102a2e"
server_key_iv: "063bd772c7b8906d56c1865b48228898"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.633039 6369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "fbf00095b6844884995dad5b9a514212"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "25d3732117db1d5c990e703aa2102a2e"
server_key_iv: "063bd772c7b8906d56c1865b48228898"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.634545 6369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:06.635663 6383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.635947 6369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:06.636070 6369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "fbf00095b6844884995dad5b9a514212"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "25d3732117db1d5c990e703aa2102a2e"
server_key_iv: "063bd772c7b8906d56c1865b48228898"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.636164 6369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:06.639933 6369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:06.640252 6369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:06.640410 6369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:06.644870 6369 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:45545
I20260807 15:02:06.644949 6439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:45545 every 8 connection(s)
I20260807 15:02:06.645313 6369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:06.646093 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:06.647557 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6369
I20260807 15:02:06.647671 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:06.647854 1535 external_mini_cluster.cc:1428] Setting key 0ff9590b3df13776b3245a10883a0004
I20260807 15:02:06.650552 6441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: Bootstrap starting.
I20260807 15:02:06.650841 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.648537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.1.127.254:45545)
0807 15:02:06.648754 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.648762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.648789 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:02:06.649252 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.649259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.649305 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.649431 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.650202 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.650205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.650206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:06.650210 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.650250 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:06.650261 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.650318 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.650421 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.650501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260807 15:02:06.651371 6441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:06.651681 6441 log.cc:824] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:06.652438 6441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: No bootstrap required, opened a new log
I20260807 15:02:06.653918 6441 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } }
I20260807 15:02:06.654076 6441 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:06.654101 6441 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf00095b6844884995dad5b9a514212, State: Initialized, Role: FOLLOWER
I20260807 15:02:06.654235 6441 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } }
I20260807 15:02:06.654335 6441 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:06.654359 6441 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:06.654431 6441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:06.655643 6441 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } }
I20260807 15:02:06.655743 6441 leader_election.cc:302] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbf00095b6844884995dad5b9a514212; no voters:
I20260807 15:02:06.655989 6441 leader_election.cc:288] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:06.656014 6446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:06.656175 6446 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [term 1 LEADER]: Becoming Leader. State: Replica: fbf00095b6844884995dad5b9a514212, State: Running, Role: LEADER
I20260807 15:02:06.656262 6446 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } }
I20260807 15:02:06.656327 6441 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:06.656634 6447 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbf00095b6844884995dad5b9a514212. Latest consensus state: current_term: 1 leader_uuid: "fbf00095b6844884995dad5b9a514212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } } }
I20260807 15:02:06.656633 6448 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbf00095b6844884995dad5b9a514212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbf00095b6844884995dad5b9a514212" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45545 } } }
I20260807 15:02:06.656873 6447 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:06.656894 6448 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:06.657375 6457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:06.657887 6457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:06.659654 6457 catalog_manager.cc:1347] Generated new cluster ID: 988357c8b4424defbffea47ea6b25946
I20260807 15:02:06.659710 6457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:06.669356 6457 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:06.669574 6457 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:06.676030 6457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: Generated new TSK 0
I20260807 15:02:06.676275 6457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:06.685516 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--builtin_ntp_servers=127.1.127.212:40321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:02:06.778805 6468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:06.779100 6468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:06.779145 6468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:06.780534 6468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:06.780622 6468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:06.780668 6468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:06.782269 6468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:06.782604 6468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:06.782987 6468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:06.789556 6474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.789525 6476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.789574 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.790331 6468 server_base.cc:1034] running on GCE node
I20260807 15:02:06.790643 6468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:06.791087 6468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:06.792268 6468 hybrid_clock.cc:648] HybridClock initialized: now 1786114926792240 us; error 43 us; skew 500 ppm
I20260807 15:02:06.793819 6468 webserver.cc:460] Webserver started at http://127.1.127.193:44395/ using document root <none> and password file <none>
I20260807 15:02:06.794178 6468 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:06.794253 6468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:06.794418 6468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:06.795763 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "141ae94ba4de47ea80c857e93e9d9938"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "1c8cd9a98a6d464a9528275a94e62be1"
server_key_iv: "865cf0e10d7f1311d864908471227906"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.796181 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "141ae94ba4de47ea80c857e93e9d9938"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "1c8cd9a98a6d464a9528275a94e62be1"
server_key_iv: "865cf0e10d7f1311d864908471227906"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.797704 6468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:06.798789 6483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.799106 6468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:06.799235 6468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "141ae94ba4de47ea80c857e93e9d9938"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "1c8cd9a98a6d464a9528275a94e62be1"
server_key_iv: "865cf0e10d7f1311d864908471227906"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.799371 6468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:06.803071 6468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:06.803395 6468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:06.803555 6468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:06.803783 6468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:06.804117 6468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:06.804188 6468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.804224 6468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:06.804242 6468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.810623 6468 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:39023
I20260807 15:02:06.810842 6600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:39023 every 8 connection(s)
I20260807 15:02:06.811131 6468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:06.814666 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6468
I20260807 15:02:06.814662 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.812036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:59335 (local address 127.1.127.254:45545)
0807 15:02:06.812095 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.812097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.812380 (+ 283us) server_negotiation.cc:407] Connection header received
0807 15:02:06.812832 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.812833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.812839 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.812906 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.814234 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.814235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.814235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:06.814240 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.814257 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:06.814261 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.814324 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.814584 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.814603 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:02:06.814823 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:06.815011 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.811981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45545 (local address 127.1.127.193:59335)
0807 15:02:06.812174 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.812216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.812707 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.812962 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.812966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.812975 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:06.814042 (+ 1067us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:06.814042 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:06.814075 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:06.814109 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:06.814468 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:06.814477 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:06.814481 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.814575 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.814660 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260807 15:02:06.815153 1535 external_mini_cluster.cc:1428] Setting key 36a6f383a0476c60bf020d70becc01cb
I20260807 15:02:06.815819 6601 heartbeater.cc:347] Connected to a master server at 127.1.127.254:45545
I20260807 15:02:06.815989 6601 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:06.816226 6601 heartbeater.cc:511] Master 127.1.127.254:45545 requested a full tablet report, sending...
I20260807 15:02:06.816633 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--builtin_ntp_servers=127.1.127.212:40321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:02:06.816900 6401 ts_manager.cc:194] Registered new tserver with Master: 141ae94ba4de47ea80c857e93e9d9938 (127.1.127.193:39023)
W20260807 15:02:06.907454 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:06.907688 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:06.907712 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:06.908861 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:06.908896 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:06.908922 6605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:06.910235 6605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:06.910444 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:06.910737 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:06.916692 6615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:06.916934 6612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.917017 6605 server_base.cc:1034] running on GCE node
W20260807 15:02:06.916692 6611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:06.917333 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:06.917685 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:06.918870 6605 hybrid_clock.cc:648] HybridClock initialized: now 1786114926918874 us; error 85 us; skew 500 ppm
I20260807 15:02:06.920625 6605 webserver.cc:460] Webserver started at http://127.1.127.194:46229/ using document root <none> and password file <none>
I20260807 15:02:06.920925 6605 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:06.921008 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:06.921202 6605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:06.922544 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "a8bf595a5ef647608ce36b9a23219539"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "efacd5eb2cac9668f22b653323fb8287"
server_key_iv: "e01646b71584aa2a50788930cd2e5505"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.923056 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "a8bf595a5ef647608ce36b9a23219539"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "efacd5eb2cac9668f22b653323fb8287"
server_key_iv: "e01646b71584aa2a50788930cd2e5505"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.924674 6605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:06.925578 6621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.925800 6605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:06.925930 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "a8bf595a5ef647608ce36b9a23219539"
format_stamp: "Formatted at 2026-08-07 15:02:06 on dist-test-slave-8tdl"
server_key: "efacd5eb2cac9668f22b653323fb8287"
server_key_iv: "e01646b71584aa2a50788930cd2e5505"
server_key_version: "encryptionkey@0"
I20260807 15:02:06.926018 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:06.930087 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:06.930398 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:06.930565 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:06.930843 6605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:06.931205 6605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:06.931264 6605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.931310 6605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:06.931341 6605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:06.938529 6605 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:37321
I20260807 15:02:06.938621 6738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:37321 every 8 connection(s)
I20260807 15:02:06.939101 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:06.942582 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.940112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:55543 (local address 127.1.127.254:45545)
0807 15:02:06.940170 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:06.940173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:06.940421 (+ 248us) server_negotiation.cc:407] Connection header received
0807 15:02:06.940948 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:06.940949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:06.940955 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:06.941049 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:06.942338 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:06.942339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:06.942339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:06.942344 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:06.942361 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:06.942365 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:06.942424 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:06.942494 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:02:06.942508 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:02:06.942864 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:06.940039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45545 (local address 127.1.127.194:55543)
0807 15:02:06.940277 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:06.940318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:06.940837 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:06.941083 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:06.941087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:06.941097 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:06.942159 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:06.942160 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:06.942193 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:06.942227 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:06.942421 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:06.942424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:06.942427 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:06.942491 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:02:06.942550 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
I20260807 15:02:06.942967 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6605
I20260807 15:02:06.943102 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:06.943360 1535 external_mini_cluster.cc:1428] Setting key c586ffc10686bc42d8014f1909d1a8ad
I20260807 15:02:06.943601 6739 heartbeater.cc:347] Connected to a master server at 127.1.127.254:45545
I20260807 15:02:06.943737 6739 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:06.943923 6739 heartbeater.cc:511] Master 127.1.127.254:45545 requested a full tablet report, sending...
I20260807 15:02:06.944382 6401 ts_manager.cc:194] Registered new tserver with Master: a8bf595a5ef647608ce36b9a23219539 (127.1.127.194:37321)
I20260807 15:02:06.944542 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--builtin_ntp_servers=127.1.127.212:40321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:02:07.035313 6742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:07.035557 6742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:07.035585 6742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:07.036880 6742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:07.036921 6742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:07.036949 6742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:07.038596 6742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:45545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:07.038936 6742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:07.039279 6742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:07.045157 6749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.045193 6752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.045177 6750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:07.045747 6742 server_base.cc:1034] running on GCE node
I20260807 15:02:07.046047 6742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:07.046286 6742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:07.047434 6742 hybrid_clock.cc:648] HybridClock initialized: now 1786114927047417 us; error 34 us; skew 500 ppm
I20260807 15:02:07.048848 6742 webserver.cc:460] Webserver started at http://127.1.127.195:45977/ using document root <none> and password file <none>
I20260807 15:02:07.049189 6742 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:07.049250 6742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:07.049428 6742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:07.050664 6742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "f966f7c2318b41459f859566d452bf76"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "dac4a8ac0bb13e7be21298135a70f980"
server_key_iv: "d399a50586712db53e97ed2f23be0fa1"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.051168 6742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "f966f7c2318b41459f859566d452bf76"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "dac4a8ac0bb13e7be21298135a70f980"
server_key_iv: "d399a50586712db53e97ed2f23be0fa1"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.052802 6742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:02:07.053895 6758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.054243 6742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:07.054375 6742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "f966f7c2318b41459f859566d452bf76"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "dac4a8ac0bb13e7be21298135a70f980"
server_key_iv: "d399a50586712db53e97ed2f23be0fa1"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.054463 6742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:07.058471 6742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:07.058841 6742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:07.059006 6742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:07.059235 6742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:07.059590 6742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:07.059646 6742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.059689 6742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:07.059720 6742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.066088 6742 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38879
I20260807 15:02:07.066130 6872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38879 every 8 connection(s)
I20260807 15:02:07.066500 6742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:07.070079 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.067372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:49913 (local address 127.1.127.254:45545)
0807 15:02:07.067486 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.067489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.067718 (+ 229us) server_negotiation.cc:407] Connection header received
0807 15:02:07.068356 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.068357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.068364 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.068444 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.069687 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.069689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.069690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.069695 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.069720 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.069724 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.069796 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.069982 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.069999 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:02:07.070315 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.067332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45545 (local address 127.1.127.195:49913)
0807 15:02:07.067573 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.067634 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.068226 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.068522 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.068526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.068536 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.069496 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.069496 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.069535 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.069568 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.069795 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.069802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.069805 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.069993 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.070067 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260807 15:02:07.070786 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6742
I20260807 15:02:07.070890 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:07.071019 6873 heartbeater.cc:347] Connected to a master server at 127.1.127.254:45545
I20260807 15:02:07.071141 6873 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:07.071170 1535 external_mini_cluster.cc:1428] Setting key f0ee8286219b1451c838b239705ad3aa
I20260807 15:02:07.071308 6873 heartbeater.cc:511] Master 127.1.127.254:45545 requested a full tablet report, sending...
I20260807 15:02:07.071765 6401 ts_manager.cc:194] Registered new tserver with Master: f966f7c2318b41459f859566d452bf76 (127.1.127.195:38879)
I20260807 15:02:07.072216 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:02:07.075026 6444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.073627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.1.127.254:45545)
0807 15:02:07.073687 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.073689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.073831 (+ 142us) server_negotiation.cc:407] Connection header received
0807 15:02:07.073891 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.073892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.073897 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.073952 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.074625 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.074626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.074627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.074631 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.074645 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.074649 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.074782 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.074841 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.074927 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:02:07.077400 6402 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:07.079522 6402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:07.084736 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.082642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38879 (local address 127.0.0.1:50386)
0807 15:02:07.083001 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.083036 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.083105 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.083456 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.083463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.083471 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.084075 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.084076 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.084096 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.084125 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.084492 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.084500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.084504 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.084561 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.084578 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"thread_start_us":131,"threads_started":1}
I20260807 15:02:07.084810 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.083325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.1.127.193:39023)
0807 15:02:07.083582 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.083586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.083597 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:07.083626 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.083627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.083638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.083915 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.084386 (+ 471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.084393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.084394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.084398 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.084429 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.084451 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.084681 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.084706 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.084722 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":102,"threads_started":1}
I20260807 15:02:07.084884 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.082642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39023 (local address 127.0.0.1:44690)
0807 15:02:07.082908 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.083071 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.083132 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.083761 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.083762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.083764 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.084266 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.084267 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.084275 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.084289 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.084562 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.084563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.084564 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.084597 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.084608 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":130,"threads_started":1}
I20260807 15:02:07.085155 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.083084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.1.127.195:38879)
0807 15:02:07.083269 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.083275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.083288 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:02:07.083320 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.083322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.083333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.083769 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.084305 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.084308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.084309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.084312 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.084367 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.084384 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.084658 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.084686 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.084705 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260807 15:02:07.085774 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.082654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37321 (local address 127.0.0.1:54560)
0807 15:02:07.083065 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.083077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.083132 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.084271 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.084272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.084275 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.085094 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.085095 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.085101 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.085113 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.085366 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.085367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.085368 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.085505 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.085520 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":140,"threads_started":1}
I20260807 15:02:07.085707 6532 tablet_service.cc:1467] Processing CreateTablet for tablet b55b7b5b41794ac79106b78146230bf6 (DEFAULT_TABLE table=test-table [id=29444692f2d64b8cadb8deb9e1358d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:07.085780 6806 tablet_service.cc:1467] Processing CreateTablet for tablet b55b7b5b41794ac79106b78146230bf6 (DEFAULT_TABLE table=test-table [id=29444692f2d64b8cadb8deb9e1358d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:07.086032 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.082864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.1.127.194:37321)
0807 15:02:07.083282 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.083287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.083297 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:07.083333 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.083335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.083343 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.083446 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.085185 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.085194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.085195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.085200 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.085240 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.085275 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.085338 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.085665 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.085687 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":99,"threads_started":1}
I20260807 15:02:07.086126 6532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55b7b5b41794ac79106b78146230bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:07.086126 6806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55b7b5b41794ac79106b78146230bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:07.086628 6669 tablet_service.cc:1467] Processing CreateTablet for tablet b55b7b5b41794ac79106b78146230bf6 (DEFAULT_TABLE table=test-table [id=29444692f2d64b8cadb8deb9e1358d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:07.087065 6669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55b7b5b41794ac79106b78146230bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:07.089876 6887 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Bootstrap starting.
I20260807 15:02:07.090296 6888 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Bootstrap starting.
I20260807 15:02:07.090866 6887 tablet_bootstrap.cc:654] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:07.090864 6889 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: Bootstrap starting.
I20260807 15:02:07.091204 6888 tablet_bootstrap.cc:654] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:07.091212 6887 log.cc:824] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:07.091563 6888 log.cc:824] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:07.091617 6889 tablet_bootstrap.cc:654] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:07.091953 6889 log.cc:824] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:07.092207 6887 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: No bootstrap required, opened a new log
I20260807 15:02:07.092290 6887 ts_tablet_manager.cc:1397] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 15:02:07.092409 6888 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: No bootstrap required, opened a new log
I20260807 15:02:07.092485 6888 ts_tablet_manager.cc:1397] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:02:07.092742 6889 tablet_bootstrap.cc:492] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: No bootstrap required, opened a new log
I20260807 15:02:07.092834 6889 ts_tablet_manager.cc:1397] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:07.095211 6889 raft_consensus.cc:348] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.095448 6889 raft_consensus.cc:374] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:07.095510 6889 raft_consensus.cc:729] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 141ae94ba4de47ea80c857e93e9d9938, State: Initialized, Role: FOLLOWER
I20260807 15:02:07.095496 6887 raft_consensus.cc:348] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.095497 6888 raft_consensus.cc:348] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.095678 6888 raft_consensus.cc:374] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:07.095678 6887 raft_consensus.cc:374] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:07.095650 6889 consensus_queue.cc:260] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.095734 6887 raft_consensus.cc:729] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8bf595a5ef647608ce36b9a23219539, State: Initialized, Role: FOLLOWER
I20260807 15:02:07.095736 6888 raft_consensus.cc:729] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f966f7c2318b41459f859566d452bf76, State: Initialized, Role: FOLLOWER
I20260807 15:02:07.095881 6887 consensus_queue.cc:260] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.095881 6888 consensus_queue.cc:260] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.096011 6889 ts_tablet_manager.cc:1428] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:02:07.096156 6601 heartbeater.cc:503] Master 127.1.127.254:45545 was elected leader, sending a full tablet report...
I20260807 15:02:07.096175 6887 ts_tablet_manager.cc:1428] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 15:02:07.096175 6888 ts_tablet_manager.cc:1428] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260807 15:02:07.096307 6873 heartbeater.cc:503] Master 127.1.127.254:45545 was elected leader, sending a full tablet report...
I20260807 15:02:07.096341 6739 heartbeater.cc:503] Master 127.1.127.254:45545 was elected leader, sending a full tablet report...
W20260807 15:02:07.189811 6740 tablet.cc:2369] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:07.245579 6894 raft_consensus.cc:482] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:07.245723 6894 raft_consensus.cc:504] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.246162 6894 leader_election.cc:288] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f966f7c2318b41459f859566d452bf76 (127.1.127.195:38879), a8bf595a5ef647608ce36b9a23219539 (127.1.127.194:37321)
I20260807 15:02:07.247699 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.246598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:44849 (local address 127.1.127.194:37321)
0807 15:02:07.246643 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.246645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.246734 (+ 89us) server_negotiation.cc:407] Connection header received
0807 15:02:07.246812 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.246813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.246818 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.246984 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.247444 (+ 460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.247446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.247446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.247450 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.247469 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.247474 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.247531 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.247624 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.247638 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:02:07.247694 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.246515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37737 (local address 127.1.127.195:38879)
0807 15:02:07.246585 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.246587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.246598 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:07.246632 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.246633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.246639 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.246751 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.247379 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.247380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.247381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.247385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.247425 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.247430 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.247495 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.247602 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.247619 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:02:07.247684 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.246391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38879 (local address 127.1.127.193:37737)
0807 15:02:07.246482 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.246498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.246551 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.246759 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.246760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.246763 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.247277 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.247277 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.247287 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.247303 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.247523 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.247525 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.247525 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.247582 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.247598 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:02:07.247697 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.246428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37321 (local address 127.1.127.193:44849)
0807 15:02:07.246641 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.246656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.246743 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.246995 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.246996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.247013 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:07.247342 (+ 329us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:07.247343 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:07.247348 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:07.247359 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:07.247538 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:07.247538 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:07.247539 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:02:07.247612 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:02:07.247622 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":107,"threads_started":1}
I20260807 15:02:07.248096 6689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55b7b5b41794ac79106b78146230bf6" candidate_uuid: "141ae94ba4de47ea80c857e93e9d9938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8bf595a5ef647608ce36b9a23219539" is_pre_election: true
I20260807 15:02:07.248275 6689 raft_consensus.cc:2454] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 141ae94ba4de47ea80c857e93e9d9938 in term 0.
I20260807 15:02:07.248664 6485 leader_election.cc:302] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 141ae94ba4de47ea80c857e93e9d9938, a8bf595a5ef647608ce36b9a23219539; no voters:
I20260807 15:02:07.248708 6826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55b7b5b41794ac79106b78146230bf6" candidate_uuid: "141ae94ba4de47ea80c857e93e9d9938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f966f7c2318b41459f859566d452bf76" is_pre_election: true
I20260807 15:02:07.248859 6826 raft_consensus.cc:2454] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 141ae94ba4de47ea80c857e93e9d9938 in term 0.
I20260807 15:02:07.248852 6894 raft_consensus.cc:2781] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:07.248955 6894 raft_consensus.cc:482] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:07.248998 6894 raft_consensus.cc:3037] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:07.249881 6894 raft_consensus.cc:504] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.250072 6894 leader_election.cc:288] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [CANDIDATE]: Term 1 election: Requested vote from peers f966f7c2318b41459f859566d452bf76 (127.1.127.195:38879), a8bf595a5ef647608ce36b9a23219539 (127.1.127.194:37321)
I20260807 15:02:07.250255 6689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55b7b5b41794ac79106b78146230bf6" candidate_uuid: "141ae94ba4de47ea80c857e93e9d9938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8bf595a5ef647608ce36b9a23219539"
I20260807 15:02:07.250404 6689 raft_consensus.cc:3037] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:07.250449 6826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55b7b5b41794ac79106b78146230bf6" candidate_uuid: "141ae94ba4de47ea80c857e93e9d9938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f966f7c2318b41459f859566d452bf76"
I20260807 15:02:07.250576 6826 raft_consensus.cc:3037] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:07.251266 6689 raft_consensus.cc:2454] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 141ae94ba4de47ea80c857e93e9d9938 in term 1.
I20260807 15:02:07.251433 6826 raft_consensus.cc:2454] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 141ae94ba4de47ea80c857e93e9d9938 in term 1.
I20260807 15:02:07.251469 6485 leader_election.cc:302] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 141ae94ba4de47ea80c857e93e9d9938, a8bf595a5ef647608ce36b9a23219539; no voters:
I20260807 15:02:07.251561 6894 raft_consensus.cc:2781] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:07.251744 6894 raft_consensus.cc:686] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [term 1 LEADER]: Becoming Leader. State: Replica: 141ae94ba4de47ea80c857e93e9d9938, State: Running, Role: LEADER
I20260807 15:02:07.251864 6894 consensus_queue.cc:237] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } }
I20260807 15:02:07.252821 6402 catalog_manager.cc:5495] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 reported cstate change: term changed from 0 to 1, leader changed from <none> to 141ae94ba4de47ea80c857e93e9d9938 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "141ae94ba4de47ea80c857e93e9d9938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "141ae94ba4de47ea80c857e93e9d9938" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:07.273694 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.272163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.1.127.193:39023)
0807 15:02:07.272246 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.272248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.272255 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:07.272296 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.272297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.272313 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.272381 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.273389 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.273390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.273391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.273396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.273421 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.273426 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.273522 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.273588 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.273603 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:02:07.275866 6826 raft_consensus.cc:1261] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 1 FOLLOWER]: Refusing update from remote peer 141ae94ba4de47ea80c857e93e9d9938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:07.275866 6689 raft_consensus.cc:1261] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 1 FOLLOWER]: Refusing update from remote peer 141ae94ba4de47ea80c857e93e9d9938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:07.276269 6894 consensus_queue.cc:1035] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8bf595a5ef647608ce36b9a23219539" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:07.276445 6898 consensus_queue.cc:1035] T b55b7b5b41794ac79106b78146230bf6 P 141ae94ba4de47ea80c857e93e9d9938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f966f7c2318b41459f859566d452bf76" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:07.280702 6904 mvcc.cc:205] Tried to move back new op lower bound from 7315926742118813696 to 7315926742024552448. Current Snapshot: MvccSnapshot[applied={T|T < 7315926742118813696}]
I20260807 15:02:07.280952 6903 mvcc.cc:205] Tried to move back new op lower bound from 7315926742118813696 to 7315926742024552448. Current Snapshot: MvccSnapshot[applied={T|T < 7315926742118813696}]
I20260807 15:02:07.282439 6402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:07.282567 6402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:07.283481 6402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbf00095b6844884995dad5b9a514212: Sending DeleteTablet for 3 replicas of tablet b55b7b5b41794ac79106b78146230bf6
I20260807 15:02:07.283887 6806 tablet_service.cc:1514] Processing DeleteTablet for tablet b55b7b5b41794ac79106b78146230bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:07 UTC) from {username='slave'} at 127.0.0.1:50386
I20260807 15:02:07.283869 6532 tablet_service.cc:1514] Processing DeleteTablet for tablet b55b7b5b41794ac79106b78146230bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:07 UTC) from {username='slave'} at 127.0.0.1:44690
I20260807 15:02:07.283883 6669 tablet_service.cc:1514] Processing DeleteTablet for tablet b55b7b5b41794ac79106b78146230bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:07 UTC) from {username='slave'} at 127.0.0.1:54560
I20260807 15:02:07.284237 6912 tablet_replica.cc:330] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: stopping tablet replica
I20260807 15:02:07.284361 6912 raft_consensus.cc:2229] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:07.284498 6912 raft_consensus.cc:2258] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:07.284575 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6468
I20260807 15:02:07.286046 6912 ts_tablet_manager.cc:1905] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:07.288564 6912 ts_tablet_manager.cc:1918] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:07.288678 6912 log.cc:1196] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/b55b7b5b41794ac79106b78146230bf6
I20260807 15:02:07.289064 6912 ts_tablet_manager.cc:1939] T b55b7b5b41794ac79106b78146230bf6 P f966f7c2318b41459f859566d452bf76: Deleting consensus metadata
I20260807 15:02:07.289789 6914 tablet_replica.cc:330] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: stopping tablet replica
I20260807 15:02:07.289983 6914 raft_consensus.cc:2229] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:07.290082 6387 catalog_manager.cc:4841] TS f966f7c2318b41459f859566d452bf76 (127.1.127.195:38879): tablet b55b7b5b41794ac79106b78146230bf6 (table test-table [id=29444692f2d64b8cadb8deb9e1358d36]) successfully deleted
I20260807 15:02:07.290153 6914 raft_consensus.cc:2258] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:07.290947 6914 ts_tablet_manager.cc:1905] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:07.293205 6914 ts_tablet_manager.cc:1918] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:07.293308 6914 log.cc:1196] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/b55b7b5b41794ac79106b78146230bf6
I20260807 15:02:07.293614 6914 ts_tablet_manager.cc:1939] T b55b7b5b41794ac79106b78146230bf6 P a8bf595a5ef647608ce36b9a23219539: Deleting consensus metadata
I20260807 15:02:07.294659 6385 catalog_manager.cc:4841] TS a8bf595a5ef647608ce36b9a23219539 (127.1.127.194:37321): tablet b55b7b5b41794ac79106b78146230bf6 (table test-table [id=29444692f2d64b8cadb8deb9e1358d36]) successfully deleted
W20260807 15:02:07.296370 6384 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:39023 (error 108)
I20260807 15:02:07.296640 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6605
I20260807 15:02:07.298334 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.296918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39023 (local address 127.0.0.1:44694)
0807 15:02:07.298241 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.298299 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1298}
W20260807 15:02:07.298497 6384 catalog_manager.cc:4584] TS 141ae94ba4de47ea80c857e93e9d9938 (127.1.127.193:39023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b55b7b5b41794ac79106b78146230bf6: Network error: Client connection negotiation failed: client connection to 127.1.127.193:39023: connect: Connection refused (error 111)
I20260807 15:02:07.303514 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6742
I20260807 15:02:07.312810 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6369
2026-08-07T15:02:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (811 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 15:02:07.335264 1535 mini_oidc.cc:152] Starting JWKS server
I20260807 15:02:07.336638 1535 webserver.cc:460] Webserver started at https://127.0.0.1:37037/ using document root <none> and password file <none>
I20260807 15:02:07.336784 1535 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:02:07.337003 1535 webserver.cc:460] Webserver started at http://127.0.0.1:34089/ using document root <none> and password file <none>
2026-08-07T15:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:07Z Disabled control of system clock
I20260807 15:02:07.347609 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39457
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:39457
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37037/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/testchainca.cert with env {}
W20260807 15:02:07.438298 6921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:07.438551 6921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:07.438578 6921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:07.439911 6921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:07.439975 6921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:07.439989 6921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:07.440002 6921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:07.440021 6921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:02:07.440033 6921 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:02:07.440045 6921 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37037/jwks.json
I20260807 15:02:07.441874 6921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:39457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37037/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:07.442142 6921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:07.442324 6921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:07.447567 6928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.447647 6929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.447584 6931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:07.447804 6921 server_base.cc:1034] running on GCE node
I20260807 15:02:07.448194 6921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:07.448580 6921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:07.449787 6921 hybrid_clock.cc:648] HybridClock initialized: now 1786114927449772 us; error 32 us; skew 500 ppm
I20260807 15:02:07.451722 6921 webserver.cc:460] Webserver started at http://127.1.127.254:43437/ using document root <none> and password file <none>
I20260807 15:02:07.452102 6921 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:07.452198 6921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:07.452386 6921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:07.453748 6921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "d6bd972b381a4fac8dd8b4edabbe01d5"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ea2a08977dddfe82ddb14706a235685c"
server_key_iv: "52be5b7def296d10acb965a839b98e08"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.454188 6921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "d6bd972b381a4fac8dd8b4edabbe01d5"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ea2a08977dddfe82ddb14706a235685c"
server_key_iv: "52be5b7def296d10acb965a839b98e08"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.455929 6921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 15:02:07.456918 6937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.457192 6921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:07.457326 6921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "d6bd972b381a4fac8dd8b4edabbe01d5"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ea2a08977dddfe82ddb14706a235685c"
server_key_iv: "52be5b7def296d10acb965a839b98e08"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.457429 6921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:07.471221 6921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:07.471652 6921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:07.471928 6921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:07.477164 6921 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:39457
I20260807 15:02:07.477236 6993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:39457 every 8 connection(s)
I20260807 15:02:07.477671 6921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:07.479110 6994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:07.482246 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: Bootstrap starting.
I20260807 15:02:07.483137 6994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:07.483454 6994 log.cc:824] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:07.483917 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 6921
I20260807 15:02:07.484056 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:07.484282 1535 external_mini_cluster.cc:1428] Setting key c00022bd57f7d4a8f79b6d2c881f4276
I20260807 15:02:07.484416 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: No bootstrap required, opened a new log
I20260807 15:02:07.486532 6994 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } }
I20260807 15:02:07.486783 6994 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:07.486825 6994 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bd972b381a4fac8dd8b4edabbe01d5, State: Initialized, Role: FOLLOWER
I20260807 15:02:07.487012 6994 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } }
I20260807 15:02:07.487138 6994 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:07.487205 6994 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:07.487270 6994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:07.487931 6994 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } }
I20260807 15:02:07.488083 6994 leader_election.cc:302] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6bd972b381a4fac8dd8b4edabbe01d5; no voters:
I20260807 15:02:07.488313 6994 leader_election.cc:288] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:07.488351 6999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:07.488476 6999 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [term 1 LEADER]: Becoming Leader. State: Replica: d6bd972b381a4fac8dd8b4edabbe01d5, State: Running, Role: LEADER
I20260807 15:02:07.488631 6994 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:07.488648 6999 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } }
I20260807 15:02:07.490728 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } } }
I20260807 15:02:07.490912 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6bd972b381a4fac8dd8b4edabbe01d5. Latest consensus state: current_term: 1 leader_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bd972b381a4fac8dd8b4edabbe01d5" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39457 } } }
I20260807 15:02:07.491065 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:07.491080 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5 [sys.catalog]: This master's current role is: LEADER
W20260807 15:02:07.491834 7016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:02:07.491915 7016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:02:07.492017 7017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:07.492226 7017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:07.492892 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.484948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.1.127.254:39457)
0807 15:02:07.485170 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.485177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.485204 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:02:07.486549 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.486562 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.486642 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.486899 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.487226 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.488768 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.489913 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.490462 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.490590 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.491927 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.491931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.491932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.492148 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.492193 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.492209 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.492271 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.492562 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.492650 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260807 15:02:07.493927 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:02:07.494885 7017 catalog_manager.cc:1347] Generated new cluster ID: 069b3b7ae3c247488e9feddc87de703b
I20260807 15:02:07.494946 7017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:07.507366 7017 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:07.508049 7017 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:07.515851 7017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6bd972b381a4fac8dd8b4edabbe01d5: Generated new TSK 0
I20260807 15:02:07.516124 7017 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:02:07.552392 7027 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 15:02:07.552703 7027 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:02:07.552742 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.527281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.1.127.254:39457)
0807 15:02:07.527335 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.527342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.527529 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:02:07.527620 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.527621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.527635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.527709 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:02:07.527941 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.528455 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.529811 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.530033 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.530103 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.530406 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:02:07.552336 (+ 21930us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:02:07.552648 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42936: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:02:07.553146 6998 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42936: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 15:02:07.553246 7026 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:07.527136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39457 (local address 127.0.0.1:42936)
0807 15:02:07.527388 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.527404 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.527507 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.527741 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.527742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.527744 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:02:07.527832 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:07.527836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.528570 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.528571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:07.529689 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:07.529693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.530128 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.530130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:07.530208 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.530242 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:02:07.552558 (+ 22316us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:02:07.552706 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.127.254:39457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.127.254:39457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37037/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 15:02:07.553881 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 6921
2026-08-07T15:02:07Z chronyd exiting
I20260807 15:02:07.592311 1535 test_util.cc:175] -----------------------------------------------
I20260807 15:02:07.592425 1535 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786114902562993-1535-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 15:02:07.593438 1535 mini_oidc.cc:152] Starting JWKS server
I20260807 15:02:07.594257 1535 webserver.cc:460] Webserver started at https://127.0.0.1:36739/ using document root <none> and password file <none>
I20260807 15:02:07.594339 1535 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:02:07.594533 1535 webserver.cc:460] Webserver started at http://127.0.0.1:33693/ using document root <none> and password file <none>
2026-08-07T15:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:07Z Disabled control of system clock
I20260807 15:02:07.604653 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36891
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:36891
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36739/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/testchainca.cert with env {}
W20260807 15:02:07.696117 7033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:07.696405 7033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:07.696446 7033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:07.697697 7033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:07.697768 7033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:07.697790 7033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:07.697813 7033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:07.697839 7033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:02:07.697865 7033 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:02:07.697892 7033 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36739/jwks.json
I20260807 15:02:07.699638 7033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:36891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36739/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:07.699955 7033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:07.700198 7033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:07.705775 7038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.705885 7041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:07.705956 7033 server_base.cc:1034] running on GCE node
W20260807 15:02:07.706067 7039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:07.706418 7033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:07.706770 7033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:07.707958 7033 hybrid_clock.cc:648] HybridClock initialized: now 1786114927707931 us; error 43 us; skew 500 ppm
I20260807 15:02:07.709561 7033 webserver.cc:460] Webserver started at http://127.1.127.254:44391/ using document root <none> and password file <none>
I20260807 15:02:07.709934 7033 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:07.709990 7033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:07.710127 7033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:07.711488 7033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "26c0f3689ca344a9ad77151dfe36687e"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "c8b1e6c7b1d80af06e095458a27f775f"
server_key_iv: "12510e3bef3d45189513d92a79564404"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.711840 7033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "26c0f3689ca344a9ad77151dfe36687e"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "c8b1e6c7b1d80af06e095458a27f775f"
server_key_iv: "12510e3bef3d45189513d92a79564404"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.713266 7033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:07.714181 7047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.714449 7033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:07.714543 7033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "26c0f3689ca344a9ad77151dfe36687e"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "c8b1e6c7b1d80af06e095458a27f775f"
server_key_iv: "12510e3bef3d45189513d92a79564404"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.714602 7033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:07.759558 7033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:07.759914 7033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:07.760035 7033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:07.764650 7033 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:36891
I20260807 15:02:07.764734 7104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:36891 every 8 connection(s)
I20260807 15:02:07.765113 7033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:07.766167 7105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:07.768826 7105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e: Bootstrap starting.
I20260807 15:02:07.769490 7105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:07.769806 7105 log.cc:824] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:07.770612 7105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e: No bootstrap required, opened a new log
I20260807 15:02:07.770929 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7033
I20260807 15:02:07.771046 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:07.771225 1535 external_mini_cluster.cc:1428] Setting key e29bcced9bf220da44237e7288555d75
I20260807 15:02:07.772857 7105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } }
I20260807 15:02:07.773015 7105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:07.773038 7105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c0f3689ca344a9ad77151dfe36687e, State: Initialized, Role: FOLLOWER
I20260807 15:02:07.773171 7105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } }
I20260807 15:02:07.773278 7105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:07.773325 7105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:07.773406 7105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:07.774276 7105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } }
I20260807 15:02:07.774399 7105 leader_election.cc:302] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c0f3689ca344a9ad77151dfe36687e; no voters:
I20260807 15:02:07.774569 7105 leader_election.cc:288] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:07.774653 7110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:07.774884 7105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:07.774909 7110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [term 1 LEADER]: Becoming Leader. State: Replica: 26c0f3689ca344a9ad77151dfe36687e, State: Running, Role: LEADER
I20260807 15:02:07.775146 7110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } }
I20260807 15:02:07.775610 7111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c0f3689ca344a9ad77151dfe36687e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } } }
I20260807 15:02:07.775697 7111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:07.775866 7112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c0f3689ca344a9ad77151dfe36687e. Latest consensus state: current_term: 1 leader_uuid: "26c0f3689ca344a9ad77151dfe36687e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c0f3689ca344a9ad77151dfe36687e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36891 } } }
I20260807 15:02:07.775918 7112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:07.776026 7119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:07.776788 7119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:07.778731 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.771800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.1.127.254:36891)
0807 15:02:07.772087 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.772095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.772127 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:02:07.772843 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.772850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.772978 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.773148 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:07.773428 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.775274 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.776483 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.777103 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.777235 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.777948 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:07.777951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:07.777951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:07.778124 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:07.778157 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:07.778168 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:07.778221 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:07.778337 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 15:02:07.778431 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260807 15:02:07.779717 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:02:07.779752 7119 catalog_manager.cc:1347] Generated new cluster ID: 1fcc1d0c03a14be0ab68b1be22ff362a
I20260807 15:02:07.779803 7119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:07.792845 7119 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:07.793469 7119 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:07.798364 7119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c0f3689ca344a9ad77151dfe36687e: Generated new TSK 0
I20260807 15:02:07.798614 7119 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:02:07.824972 7135 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 15:02:07.825124 7135 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:02:07.825487 7109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:07.808450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.1.127.254:36891)
0807 15:02:07.808532 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:07.808534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:07.808575 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:02:07.808655 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:07.808656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:07.808670 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:07.808761 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:02:07.808926 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.809810 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.810814 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.811167 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.811248 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.811506 (+ 258us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:02:07.825195 (+ 13689us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:02:07.825403 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 15:02:07.825557 7109 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 15:02:07.825603 7134 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:07.808302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:36891 (local address 127.0.0.1:35706)
0807 15:02:07.808496 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:07.808511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:07.808585 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:07.808770 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:07.808770 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:07.808772 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:02:07.808837 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:07.808839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.809938 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.809939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:07.810637 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:07.810640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:07.811276 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:07.811277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:07.811328 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:07.811334 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:02:07.825343 (+ 14009us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:02:07.825484 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.127.254:36891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260807 15:02:07.825954 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7033
2026-08-07T15:02:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (272 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 15:02:07.865797 1535 mini_oidc.cc:152] Starting JWKS server
I20260807 15:02:07.866551 1535 webserver.cc:460] Webserver started at https://127.0.0.1:45053/ using document root <none> and password file <none>
I20260807 15:02:07.866627 1535 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:02:07.866875 1535 webserver.cc:460] Webserver started at http://127.0.0.1:41231/ using document root <none> and password file <none>
2026-08-07T15:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:07Z Disabled control of system clock
I20260807 15:02:07.876408 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39971
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:39971
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45053/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:02:07.962884 7141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:07.963135 7141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:07.963160 7141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:07.964419 7141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:07.964473 7141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:07.964485 7141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:07.964498 7141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:07.964510 7141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:02:07.964522 7141 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:02:07.964534 7141 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45053/jwks.json
I20260807 15:02:07.965983 7141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:39971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45053/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:07.966272 7141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:07.966615 7141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:07.972259 7149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.972492 7147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:07.972256 7146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:07.972627 7141 server_base.cc:1034] running on GCE node
I20260807 15:02:07.972879 7141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:07.973284 7141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:07.974462 7141 hybrid_clock.cc:648] HybridClock initialized: now 1786114927974439 us; error 47 us; skew 500 ppm
I20260807 15:02:07.976177 7141 webserver.cc:460] Webserver started at http://127.1.127.254:39505/ using document root <none> and password file <none>
I20260807 15:02:07.976454 7141 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:07.976541 7141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:07.976701 7141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:07.977864 7141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "5d81df096b4b422d8121f6888313e3d7"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ab418f56b1ff95cfa7a215a32f9f264b"
server_key_iv: "57881e20a597cd4f22523498ac27f53b"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.978246 7141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "5d81df096b4b422d8121f6888313e3d7"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ab418f56b1ff95cfa7a215a32f9f264b"
server_key_iv: "57881e20a597cd4f22523498ac27f53b"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.979773 7141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:07.980731 7155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:07.981056 7141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:07.981184 7141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "5d81df096b4b422d8121f6888313e3d7"
format_stamp: "Formatted at 2026-08-07 15:02:07 on dist-test-slave-8tdl"
server_key: "ab418f56b1ff95cfa7a215a32f9f264b"
server_key_iv: "57881e20a597cd4f22523498ac27f53b"
server_key_version: "encryptionkey@0"
I20260807 15:02:07.981302 7141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:07.995339 7141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:07.995707 7141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:07.995869 7141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:08.000402 7141 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:39971
I20260807 15:02:08.000527 7210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:39971 every 8 connection(s)
I20260807 15:02:08.000864 7141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:08.001655 7211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:08.001737 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7141
I20260807 15:02:08.001843 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:08.002068 1535 external_mini_cluster.cc:1428] Setting key 816ba57c9bd5bfe58d883f8905b50c61
I20260807 15:02:08.006026 7211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7: Bootstrap starting.
I20260807 15:02:08.006958 7211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:08.007310 7211 log.cc:824] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:08.008407 7211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7: No bootstrap required, opened a new log
I20260807 15:02:08.009634 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:08.002838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.1.127.254:39971)
0807 15:02:08.003146 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:08.003155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:08.003183 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:02:08.003858 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:08.003868 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:08.004069 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:08.004219 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:08.004600 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.005626 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.006960 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.007621 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.007779 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.008755 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:08.008760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:08.008760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:08.008935 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:08.008972 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:08.008984 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:08.009040 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:08.009184 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:02:08.009406 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":182,"threads_started":1}
I20260807 15:02:08.011432 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:02:08.011345 7211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } }
I20260807 15:02:08.011571 7211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:08.011624 7211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d81df096b4b422d8121f6888313e3d7, State: Initialized, Role: FOLLOWER
I20260807 15:02:08.011776 7211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } }
I20260807 15:02:08.011861 7211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:08.011906 7211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:08.011958 7211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:08.012777 7211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } }
I20260807 15:02:08.013114 7211 leader_election.cc:302] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d81df096b4b422d8121f6888313e3d7; no voters:
I20260807 15:02:08.013341 7211 leader_election.cc:288] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:08.013435 7216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:08.013600 7216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [term 1 LEADER]: Becoming Leader. State: Replica: 5d81df096b4b422d8121f6888313e3d7, State: Running, Role: LEADER
I20260807 15:02:08.013628 7211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:08.013696 7216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } }
I20260807 15:02:08.014431 7219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d81df096b4b422d8121f6888313e3d7. Latest consensus state: current_term: 1 leader_uuid: "5d81df096b4b422d8121f6888313e3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } } }
I20260807 15:02:08.014593 7219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:08.015021 7226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:08.015076 7218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d81df096b4b422d8121f6888313e3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d81df096b4b422d8121f6888313e3d7" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39971 } } }
I20260807 15:02:08.015172 7218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:08.015539 7226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:08.017205 7226 catalog_manager.cc:1347] Generated new cluster ID: 84b099861953490faaf37d9e22855931
I20260807 15:02:08.017267 7226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:08.028771 7226 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:08.029373 7226 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:08.040197 7226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d81df096b4b422d8121f6888313e3d7: Generated new TSK 0
I20260807 15:02:08.040426 7226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:08.054262 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:08.049709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.1.127.254:39971)
0807 15:02:08.049785 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:08.049788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:08.049890 (+ 102us) server_negotiation.cc:407] Connection header received
0807 15:02:08.049954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:08.049955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:08.049970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:08.050067 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:02:08.050314 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.051326 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.052260 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.052615 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.052694 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.052926 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:02:08.054024 (+ 1098us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45053/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:02:08.054173 (+ 149us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58840: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45053/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 15:02:08.054451 7214 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58840: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45053/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 15:02:08.056346 7242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:08.049601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39971 (local address 127.0.0.1:58840)
0807 15:02:08.049819 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:08.049829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:08.049886 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:08.050155 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:08.050157 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:08.050159 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:02:08.050224 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:08.050227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.051460 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.051461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:08.052160 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:08.052162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.052730 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.052731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:08.052791 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.052798 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:02:08.054152 (+ 1354us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45053/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:02:08.056237 (+ 2085us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.127.254:39971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45053/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1836,"thread_start_us":129,"threads_started":1}
I20260807 15:02:08.057081 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7141
E20260807 15:02:08.067660 7243 webserver.cc:533] Webserver: sslize failed:
E20260807 15:02:08.067749 7243 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T15:02:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T15:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:08Z Disabled control of system clock
I20260807 15:02:08.106895 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41557
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:43439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41557
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/testchainca.cert with env {}
W20260807 15:02:08.196116 7247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:08.196348 7247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:08.196373 7247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:08.197548 7247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:08.197588 7247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:08.197602 7247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:08.197615 7247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:08.197628 7247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:02:08.197640 7247 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:02:08.197652 7247 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 15:02:08.199174 7247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:08.199391 7247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:08.199561 7247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:08.205029 7253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:08.205015 7255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:08.205010 7252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:08.205057 7247 server_base.cc:1034] running on GCE node
I20260807 15:02:08.205543 7247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:08.205891 7247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:08.207072 7247 hybrid_clock.cc:648] HybridClock initialized: now 1786114928207044 us; error 59 us; skew 500 ppm
I20260807 15:02:08.208483 7247 webserver.cc:460] Webserver started at http://127.1.127.254:36945/ using document root <none> and password file <none>
I20260807 15:02:08.208783 7247 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:08.208860 7247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:08.209028 7247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:08.210134 7247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "2dd1b282951c45f299ed2461a29fb96d"
format_stamp: "Formatted at 2026-08-07 15:02:08 on dist-test-slave-8tdl"
server_key: "8221ca3e6b22aca3ff6091ceb2654709"
server_key_iv: "f462ec533d965b1430e88656a3280c35"
server_key_version: "encryptionkey@0"
I20260807 15:02:08.210522 7247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "2dd1b282951c45f299ed2461a29fb96d"
format_stamp: "Formatted at 2026-08-07 15:02:08 on dist-test-slave-8tdl"
server_key: "8221ca3e6b22aca3ff6091ceb2654709"
server_key_iv: "f462ec533d965b1430e88656a3280c35"
server_key_version: "encryptionkey@0"
I20260807 15:02:08.212154 7247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:08.213233 7261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:08.213507 7247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:08.213634 7247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "2dd1b282951c45f299ed2461a29fb96d"
format_stamp: "Formatted at 2026-08-07 15:02:08 on dist-test-slave-8tdl"
server_key: "8221ca3e6b22aca3ff6091ceb2654709"
server_key_iv: "f462ec533d965b1430e88656a3280c35"
server_key_version: "encryptionkey@0"
I20260807 15:02:08.213725 7247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:08.232241 7247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:08.232743 7247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:08.232940 7247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:08.237808 7247 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41557
I20260807 15:02:08.237862 7320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41557 every 8 connection(s)
I20260807 15:02:08.238299 7247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:08.239565 7321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:08.242237 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d: Bootstrap starting.
I20260807 15:02:08.243021 7321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:08.243013 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7247
I20260807 15:02:08.243139 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:08.243362 1535 external_mini_cluster.cc:1428] Setting key a80be01441088689d54abbe4984f6d23
I20260807 15:02:08.243402 7321 log.cc:824] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:08.244515 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d: No bootstrap required, opened a new log
I20260807 15:02:08.247430 7321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } }
I20260807 15:02:08.247634 7321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:08.247685 7321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dd1b282951c45f299ed2461a29fb96d, State: Initialized, Role: FOLLOWER
I20260807 15:02:08.247889 7321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } }
I20260807 15:02:08.247985 7321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:08.248031 7321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:08.248083 7321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:08.248944 7321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } }
I20260807 15:02:08.249089 7321 leader_election.cc:302] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dd1b282951c45f299ed2461a29fb96d; no voters:
I20260807 15:02:08.249300 7321 leader_election.cc:288] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:08.249369 7326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:08.249572 7326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [term 1 LEADER]: Becoming Leader. State: Replica: 2dd1b282951c45f299ed2461a29fb96d, State: Running, Role: LEADER
I20260807 15:02:08.249624 7321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:08.249719 7326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } }
I20260807 15:02:08.250345 7327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dd1b282951c45f299ed2461a29fb96d. Latest consensus state: current_term: 1 leader_uuid: "2dd1b282951c45f299ed2461a29fb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } } }
I20260807 15:02:08.250451 7327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:08.250746 7335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:08.251279 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:08.243937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.1.127.254:41557)
0807 15:02:08.244243 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:08.244253 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:08.244291 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:02:08.244981 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:08.244992 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:08.245082 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:08.245252 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:08.245554 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.247097 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.248304 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.248926 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.249122 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.250004 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:08.250009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:08.250009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:08.250236 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:08.250287 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:08.250306 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:08.250377 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:08.250882 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 15:02:08.251005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260807 15:02:08.251307 7335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:08.251902 7328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dd1b282951c45f299ed2461a29fb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd1b282951c45f299ed2461a29fb96d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41557 } } }
I20260807 15:02:08.252183 7328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:08.252403 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:02:08.253937 7335 catalog_manager.cc:1347] Generated new cluster ID: 502abf23c85b433d93806543e29b3c42
I20260807 15:02:08.253997 7335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:08.272650 7335 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:08.273479 7335 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:08.281062 7335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dd1b282951c45f299ed2461a29fb96d: Generated new TSK 0
I20260807 15:02:08.281322 7335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:08.388425 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:08.290453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.1.127.254:41557)
0807 15:02:08.290554 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:08.290556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:08.290639 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:02:08.290788 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:08.290789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:08.290802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:08.290894 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:02:08.291065 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.291591 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.292406 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.292632 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.292706 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.292903 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:02:08.388066 (+ 95163us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:02:08.388323 (+ 257us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37002: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 15:02:08.388536 7325 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37002: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 15:02:08.388550 7350 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:08.290384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41557 (local address 127.0.0.1:37002)
0807 15:02:08.290558 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:08.290570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:08.290659 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:08.290893 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:08.290893 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:08.290895 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:02:08.290973 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:08.290976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.291680 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.291682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:08.292304 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:08.292306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:08.292714 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:08.292716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:08.292781 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:08.292786 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:02:08.388260 (+ 95474us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:02:08.388415 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.127.254:41557: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260807 15:02:08.389178 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7247
2026-08-07T15:02:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (312 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (100 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:08 dist-test-slave-8tdl krb5kdc[7363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:08 dist-test-slave-8tdl krb5kdc[7363](info): set up 2 sockets
Aug 07 15:02:08 dist-test-slave-8tdl krb5kdc[7363](info): commencing operation
krb5kdc: starting...
W20260807 15:02:09.667309 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:09 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:09.810585 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35781
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:35781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:09.905373 7377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:09.905712 7377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:09.905759 7377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:09.907275 7377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:09.907359 7377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:09.907387 7377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:09.907425 7377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:09.907473 7377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:09.909322 7377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:35781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:09.909655 7377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:09.909952 7377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:09.915787 7386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:09.915783 7384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:09.915783 7383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:09.916123 7377 server_base.cc:1034] running on GCE node
I20260807 15:02:09.916429 7377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:09.916770 7377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:09.917968 7377 hybrid_clock.cc:648] HybridClock initialized: now 1786114929917957 us; error 37 us; skew 500 ppm
Aug 07 15:02:09 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114929, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:09.926293 7377 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:09.927006 7377 webserver.cc:460] Webserver started at http://127.1.127.254:35465/ using document root <none> and password file <none>
I20260807 15:02:09.927291 7377 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:09.927366 7377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:09.927536 7377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:09.928773 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "c246bd9afe824d0b9ef289e5b8407272"
format_stamp: "Formatted at 2026-08-07 15:02:09 on dist-test-slave-8tdl"
server_key: "db40ef4afa752c09bf2f75c797f7c4db"
server_key_iv: "04ae3ddb8b0bdd1f5da86f930d888b4f"
server_key_version: "encryptionkey@0"
I20260807 15:02:09.929184 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "c246bd9afe824d0b9ef289e5b8407272"
format_stamp: "Formatted at 2026-08-07 15:02:09 on dist-test-slave-8tdl"
server_key: "db40ef4afa752c09bf2f75c797f7c4db"
server_key_iv: "04ae3ddb8b0bdd1f5da86f930d888b4f"
server_key_version: "encryptionkey@0"
I20260807 15:02:09.931013 7377 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:02:09.931996 7393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:09.932233 7377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:09.932399 7377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "c246bd9afe824d0b9ef289e5b8407272"
format_stamp: "Formatted at 2026-08-07 15:02:09 on dist-test-slave-8tdl"
server_key: "db40ef4afa752c09bf2f75c797f7c4db"
server_key_iv: "04ae3ddb8b0bdd1f5da86f930d888b4f"
server_key_version: "encryptionkey@0"
I20260807 15:02:09.932503 7377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:09.949649 7377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:09.952072 7377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:09.952253 7377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:09.956842 7377 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:35781
I20260807 15:02:09.956984 7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:35781 every 8 connection(s)
I20260807 15:02:09.957312 7377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:09.958298 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:09.960752 7450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272: Bootstrap starting.
I20260807 15:02:09.961326 7450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:09.961594 7450 log.cc:824] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:09.962360 7450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272: No bootstrap required, opened a new log
I20260807 15:02:09.964143 7450 raft_consensus.cc:348] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } }
I20260807 15:02:09.964293 7450 raft_consensus.cc:374] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:09.964318 7450 raft_consensus.cc:729] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c246bd9afe824d0b9ef289e5b8407272, State: Initialized, Role: FOLLOWER
I20260807 15:02:09.964480 7450 consensus_queue.cc:260] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } }
I20260807 15:02:09.964569 7450 raft_consensus.cc:388] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:09.964594 7450 raft_consensus.cc:482] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:09.964625 7450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:09.965157 7450 raft_consensus.cc:504] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } }
I20260807 15:02:09.965246 7450 leader_election.cc:302] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c246bd9afe824d0b9ef289e5b8407272; no voters:
I20260807 15:02:09.965395 7450 leader_election.cc:288] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:09.965500 7453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:09.965709 7450 sys_catalog.cc:564] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:09.966032 7453 raft_consensus.cc:686] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [term 1 LEADER]: Becoming Leader. State: Replica: c246bd9afe824d0b9ef289e5b8407272, State: Running, Role: LEADER
I20260807 15:02:09.966138 7453 consensus_queue.cc:237] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } }
I20260807 15:02:09.966586 7455 sys_catalog.cc:455] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c246bd9afe824d0b9ef289e5b8407272. Latest consensus state: current_term: 1 leader_uuid: "c246bd9afe824d0b9ef289e5b8407272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } } }
I20260807 15:02:09.966697 7455 sys_catalog.cc:458] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:09.966754 7454 sys_catalog.cc:455] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c246bd9afe824d0b9ef289e5b8407272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c246bd9afe824d0b9ef289e5b8407272" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35781 } } }
I20260807 15:02:09.966841 7454 sys_catalog.cc:458] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:09.967003 7463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:09.967499 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7377
I20260807 15:02:09.967602 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:09.967625 7463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:09.967831 1535 external_mini_cluster.cc:1428] Setting key f16ac560d05f062395055fedbdddeef1
I20260807 15:02:09.969581 7463 catalog_manager.cc:1347] Generated new cluster ID: 98516a2c645a4207a8830b0dff28f70c
I20260807 15:02:09.969642 7463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:02:09 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:09.980193 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:09.968346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41910 (local address 127.1.127.254:35781)
0807 15:02:09.968517 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:09.968528 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:09.968707 (+ 179us) server_negotiation.cc:407] Connection header received
0807 15:02:09.968849 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:09.968856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:09.968914 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:09.969062 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:09.969898 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:09.971415 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:09.972520 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:09.973205 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:09.973324 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:09.976305 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:09.976310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:09.976311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:09.976325 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:09.978379 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:09.978868 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:09.978871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:09.978872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:09.979011 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:09.979227 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:09.979228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:09.979228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:09.979541 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:09.979699 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:09.979803 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:02:09.979951 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260807 15:02:10.002581 7463 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:10.003228 7463 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:10.006269 7463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c246bd9afe824d0b9ef289e5b8407272: Generated new TSK 0
I20260807 15:02:10.006521 7463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:10.046108 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:10.133358 7477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:10.133659 7477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:10.133703 7477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:10.135027 7477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:10.135102 7477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:10.135146 7477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:10.136895 7477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:10.137212 7477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:10.137478 7477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:10.142915 7483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.142910 7484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.142915 7487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:10.143286 7477 server_base.cc:1034] running on GCE node
I20260807 15:02:10.143525 7477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:10.143795 7477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:10.144968 7477 hybrid_clock.cc:648] HybridClock initialized: now 1786114930144949 us; error 33 us; skew 500 ppm
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.147830 7477 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:10.148460 7477 webserver.cc:460] Webserver started at http://127.1.127.193:44887/ using document root <none> and password file <none>
I20260807 15:02:10.148726 7477 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:10.148814 7477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:10.148998 7477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:10.150296 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "264d25311011438d88a4cf5db78301a0"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "2576892f8917590c864ef3f64c44fe31"
server_key_iv: "dadf2acb518863e6e9f27e705c894eaa"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.150758 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "264d25311011438d88a4cf5db78301a0"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "2576892f8917590c864ef3f64c44fe31"
server_key_iv: "dadf2acb518863e6e9f27e705c894eaa"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.152591 7477 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 15:02:10.153600 7494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.153831 7477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:10.153960 7477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "264d25311011438d88a4cf5db78301a0"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "2576892f8917590c864ef3f64c44fe31"
server_key_iv: "dadf2acb518863e6e9f27e705c894eaa"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.154045 7477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:10.168344 7477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:10.170849 7477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:10.171057 7477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:10.171342 7477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:10.171979 7477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:10.172073 7477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.172113 7477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:10.172158 7477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.179344 7477 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:40827
I20260807 15:02:10.179450 7614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:40827 every 8 connection(s)
I20260807 15:02:10.179901 7477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:10.182664 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7477
I20260807 15:02:10.182844 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:10.183058 1535 external_mini_cluster.cc:1428] Setting key 0f5ca305a33d7326ac64d9dc666ed41b
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:10.193501 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.181096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:55913 (local address 127.1.127.254:35781)
0807 15:02:10.181184 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.181186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.181512 (+ 326us) server_negotiation.cc:407] Connection header received
0807 15:02:10.182719 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.182720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.182739 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.182796 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.183680 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.184308 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.185409 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.185770 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.185852 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.188930 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.188932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.188933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.188943 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.191195 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.192141 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.192143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.192144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.192227 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.192676 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.192678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.192679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.192852 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.192922 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.193324 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.193427 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:02:10.193858 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.181060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35781 (local address 127.1.127.193:55913)
0807 15:02:10.181320 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:10.181363 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:10.182537 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:10.182906 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:10.182911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:10.183313 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:10.183583 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.183589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.184418 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.184419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.185294 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.185296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.185890 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.185891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.186042 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.186802 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:10.186818 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:10.188765 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:10.191318 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.191336 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.191340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.191979 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.192321 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.192322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.192323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.192529 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.192930 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:10.192939 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:10.193150 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 15:02:10.193293 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 15:02:10.193512 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:02:10.194506 7615 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35781
I20260807 15:02:10.194638 7615 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:10.194898 7615 heartbeater.cc:511] Master 127.1.127.254:35781 requested a full tablet report, sending...
I20260807 15:02:10.195676 7414 ts_manager.cc:194] Registered new tserver with Master: 264d25311011438d88a4cf5db78301a0 (127.1.127.193:40827)
I20260807 15:02:10.196287 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:55913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:10.217912 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:10.306835 7620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:10.307077 7620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:10.307104 7620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:10.308374 7620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:10.308432 7620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:10.308462 7620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:10.310182 7620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:10.310483 7620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:10.310799 7620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:10.316440 7628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.316381 7630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.316448 7627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:10.316839 7620 server_base.cc:1034] running on GCE node
I20260807 15:02:10.317093 7620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:10.317423 7620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:10.318614 7620 hybrid_clock.cc:648] HybridClock initialized: now 1786114930318604 us; error 52 us; skew 500 ppm
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.321501 7620 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:10.322223 7620 webserver.cc:460] Webserver started at http://127.1.127.194:35279/ using document root <none> and password file <none>
I20260807 15:02:10.322502 7620 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:10.322582 7620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:10.322773 7620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:10.323974 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "2e166b0981e54038b0b503e8f351123a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "d74691b6bce423e3c218013474ab8cb3"
server_key_iv: "a991da50aa8d7ac13225f68a846394e5"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.324388 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "2e166b0981e54038b0b503e8f351123a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "d74691b6bce423e3c218013474ab8cb3"
server_key_iv: "a991da50aa8d7ac13225f68a846394e5"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.326038 7620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:02:10.327219 7637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.327551 7620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:10.327709 7620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "2e166b0981e54038b0b503e8f351123a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "d74691b6bce423e3c218013474ab8cb3"
server_key_iv: "a991da50aa8d7ac13225f68a846394e5"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.327800 7620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:10.340626 7620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:10.343214 7620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:10.343343 7620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:10.343662 7620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:10.344031 7620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:10.344069 7620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.344091 7620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:10.344105 7620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.350450 7620 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:42425
I20260807 15:02:10.350492 7756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:42425 every 8 connection(s)
I20260807 15:02:10.350941 7620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:10.354575 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7620
I20260807 15:02:10.354715 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:10.354930 1535 external_mini_cluster.cc:1428] Setting key fd6cbb9c96ce09c9e8322b1e5e81a699
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:10.362890 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.351813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:58919 (local address 127.1.127.254:35781)
0807 15:02:10.351862 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.351864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.352149 (+ 285us) server_negotiation.cc:407] Connection header received
0807 15:02:10.353154 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.353155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.353173 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.353231 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.353854 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.354447 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.355953 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.356302 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.356384 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.358966 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.358969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.358969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.358980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.361138 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.361738 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.361740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.361740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.361815 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.362109 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.362111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.362111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.362261 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.362337 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.362655 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.362784 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:02:10.363138 7759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.351786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35781 (local address 127.1.127.194:58919)
0807 15:02:10.352021 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:10.352061 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:10.353043 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:10.353287 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:10.353291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:10.353574 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:10.353775 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.353779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.354589 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.354590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.355870 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.355873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.356401 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.356402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.356534 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.357205 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:10.357217 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:10.358856 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:10.361284 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.361297 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.361300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.361604 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.361895 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.361896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.361896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.362032 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.362339 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:10.362349 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:10.362507 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 15:02:10.362619 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 15:02:10.362856 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":104,"threads_started":1}
I20260807 15:02:10.364064 7757 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35781
I20260807 15:02:10.364229 7757 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:10.364457 7757 heartbeater.cc:511] Master 127.1.127.254:35781 requested a full tablet report, sending...
I20260807 15:02:10.364976 7414 ts_manager.cc:194] Registered new tserver with Master: 2e166b0981e54038b0b503e8f351123a (127.1.127.194:42425)
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:02:10.365592 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:58919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:10.392270 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:10.485894 7762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:10.486145 7762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:10.486171 7762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:10.487464 7762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:10.487522 7762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:10.487551 7762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:10.489180 7762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:10.489394 7762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:10.489598 7762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:10.495028 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.495030 7770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:10.495085 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:10.495421 7762 server_base.cc:1034] running on GCE node
I20260807 15:02:10.495666 7762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:10.495921 7762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:10.497203 7762 hybrid_clock.cc:648] HybridClock initialized: now 1786114930497173 us; error 36 us; skew 500 ppm
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.500360 7762 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:10.500857 7762 webserver.cc:460] Webserver started at http://127.1.127.195:39005/ using document root <none> and password file <none>
I20260807 15:02:10.501070 7762 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:10.501120 7762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:10.501225 7762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:10.502405 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "dcfe8ed26873443b9264d21bb026217a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "68dbe4e3189bfbd7d312fb4459ba2d14"
server_key_iv: "dc8d4a0ecdf23b2fb240aef4ba057408"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.502835 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "dcfe8ed26873443b9264d21bb026217a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "68dbe4e3189bfbd7d312fb4459ba2d14"
server_key_iv: "dc8d4a0ecdf23b2fb240aef4ba057408"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.504344 7762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:10.505187 7779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.505421 7762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:10.505565 7762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "dcfe8ed26873443b9264d21bb026217a"
format_stamp: "Formatted at 2026-08-07 15:02:10 on dist-test-slave-8tdl"
server_key: "68dbe4e3189bfbd7d312fb4459ba2d14"
server_key_iv: "dc8d4a0ecdf23b2fb240aef4ba057408"
server_key_version: "encryptionkey@0"
I20260807 15:02:10.505645 7762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:10.514377 7762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:10.516709 7762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:10.516877 7762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:10.517120 7762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:10.517432 7762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:10.517491 7762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.517551 7762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:10.517571 7762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:10.523710 7762 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:40677
I20260807 15:02:10.523784 7897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:40677 every 8 connection(s)
I20260807 15:02:10.524127 7762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:10.528687 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7762
I20260807 15:02:10.528816 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:10.529006 1535 external_mini_cluster.cc:1428] Setting key 42f1cec932b1d1fdf938d16e7390073e
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:10.535032 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.525065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:51845 (local address 127.1.127.254:35781)
0807 15:02:10.525130 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.525132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.525346 (+ 214us) server_negotiation.cc:407] Connection header received
0807 15:02:10.526396 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.526397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.526413 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.526467 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.527177 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.527699 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.528732 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.529018 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.529093 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.531682 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.531684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.531684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.531693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.533241 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.533777 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.533778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.533779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.533853 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.534207 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.534208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.534209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.534348 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.534429 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.534806 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.534927 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:02:10.535264 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.525044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35781 (local address 127.1.127.195:51845)
0807 15:02:10.525226 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:10.525264 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:10.526273 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:10.526542 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:10.526546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:10.526883 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:10.527085 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.527089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.527809 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.527810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.528639 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:10.528642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.529103 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.529104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:10.529253 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.529879 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:10.529887 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:10.531589 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:10.533374 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.533402 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.533405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.533676 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.533994 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:10.533995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:10.533996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:10.534125 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:10.534431 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:10.534440 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:10.534600 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 15:02:10.534777 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 15:02:10.534980 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260807 15:02:10.535900 7898 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35781
I20260807 15:02:10.536037 7898 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:10.536262 7898 heartbeater.cc:511] Master 127.1.127.254:35781 requested a full tablet report, sending...
I20260807 15:02:10.536700 7414 ts_manager.cc:194] Registered new tserver with Master: dcfe8ed26873443b9264d21bb026217a (127.1.127.195:40677)
I20260807 15:02:10.537158 7414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:51845
I20260807 15:02:10.541399 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/test-admin.keytab.
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.562067 1535 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 15:02:10.564129 7905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:10.563097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.1.127.193:40827)
0807 15:02:10.563263 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.563268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.563279 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:10.563306 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.563308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.563325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.563394 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.564027 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35400: BlockingRecv error: recv got EOF from 127.0.0.1:35400 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260807 15:02:10.564129 1535 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.565303 1535 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.566454 1535 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:02:10.576159 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.567256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.1.127.193:40827)
0807 15:02:10.567308 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.567310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.567400 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:02:10.567454 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.567455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.567466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.567525 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.567995 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.568749 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.569619 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.570013 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.570086 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.572790 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.572809 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.572810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.572817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.574675 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.575212 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.575215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.575216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.575280 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.575501 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.575502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.575503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.575679 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.575890 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.575936 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.576040 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:02:10.578383 7905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:10.577704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.1.127.193:40827)
0807 15:02:10.577775 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.577777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.577808 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:02:10.577878 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.577879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.577890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.577928 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.578307 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35430: BlockingRecv error: recv got EOF from 127.0.0.1:35430 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 15:02:10.578408 1535 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.579564 1535 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.580466 1535 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:02:10.589088 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.581196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.1.127.193:40827)
0807 15:02:10.581264 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.581265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.581280 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:02:10.581301 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.581302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.581372 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.581443 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.582034 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.582638 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.583524 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.583732 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.583794 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.586115 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.586117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.586117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.586125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.587785 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.588182 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.588183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.588183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.588238 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.588511 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.588512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.588512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.588644 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.588721 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.588820 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.588884 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 15:02:10.591050 7905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:10.590301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.1.127.193:40827)
0807 15:02:10.590400 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.590402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.590451 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:02:10.590530 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.590531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.590540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.590597 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.590994 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35458: BlockingRecv error: recv got EOF from 127.0.0.1:35458 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 15:02:10.591167 1535 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786114902562993-1535-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.592427 1535 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:10.593514 1535 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.193@KRBTEST.COM
I20260807 15:02:10.601778 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:10.594182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.1.127.193:40827)
0807 15:02:10.594255 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:10.594257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:10.594371 (+ 114us) server_negotiation.cc:407] Connection header received
0807 15:02:10.594433 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:10.594434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:10.594446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:10.594503 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:10.594991 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.595484 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.596204 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:10.596400 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:10.596462 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:10.599094 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:10.599096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:10.599096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:10.599104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:10.600721 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.601053 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.601054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.601055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.601108 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:10.601272 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:10.601273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:10.601273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:10.601373 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:10.601479 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:10.601579 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:02:10.601640 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:02:10.602583 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7477
I20260807 15:02:10.609943 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7620
I20260807 15:02:10.615610 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7762
I20260807 15:02:10.620922 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7377
2026-08-07T15:02:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2033 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7919](info): set up 2 sockets
Aug 07 15:02:10 dist-test-slave-8tdl krb5kdc[7919](info): commencing operation
krb5kdc: starting...
W20260807 15:02:11.718936 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:11 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab.
Entry for principal oryx/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab.
I20260807 15:02:11.864652 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41413
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab
--principal=oryx/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:11.962307 7933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:11.962634 7933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:11.962683 7933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:11.964159 7933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:11.964247 7933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:11.964272 7933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:11.964295 7933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:11.964322 7933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:11.966461 7933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab
--principal=oryx/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:11.966881 7933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:11.967152 7933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:11.973677 7933 server_base.cc:1034] running on GCE node
W20260807 15:02:11.973603 7941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:11.973595 7939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:11.973627 7938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:11.974308 7933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:11.974742 7933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:11.975919 7933 hybrid_clock.cc:648] HybridClock initialized: now 1786114931975904 us; error 32 us; skew 500 ppm
Aug 07 15:02:11 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:11.979113 7933 init.cc:377] Logged in from keytab as oryx/127.1.127.254@KRBTEST.COM (short username oryx)
I20260807 15:02:11.979709 7933 webserver.cc:460] Webserver started at http://127.1.127.254:42161/ using document root <none> and password file <none>
I20260807 15:02:11.979977 7933 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:11.980051 7933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:11.980206 7933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:11.981401 7933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "649e709be6fc4b46b52150eb1b3f2e6c"
format_stamp: "Formatted at 2026-08-07 15:02:11 on dist-test-slave-8tdl"
server_key: "c4e41f8d01ba658b0b52c5d9ad813b6c"
server_key_iv: "6607c41fec2000294426deb7d4f93ffc"
server_key_version: "encryptionkey@0"
I20260807 15:02:11.981830 7933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "649e709be6fc4b46b52150eb1b3f2e6c"
format_stamp: "Formatted at 2026-08-07 15:02:11 on dist-test-slave-8tdl"
server_key: "c4e41f8d01ba658b0b52c5d9ad813b6c"
server_key_iv: "6607c41fec2000294426deb7d4f93ffc"
server_key_version: "encryptionkey@0"
I20260807 15:02:11.984179 7933 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:02:11.985639 7949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:11.985930 7933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:11.986153 7933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "649e709be6fc4b46b52150eb1b3f2e6c"
format_stamp: "Formatted at 2026-08-07 15:02:11 on dist-test-slave-8tdl"
server_key: "c4e41f8d01ba658b0b52c5d9ad813b6c"
server_key_iv: "6607c41fec2000294426deb7d4f93ffc"
server_key_version: "encryptionkey@0"
I20260807 15:02:11.986249 7933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:11.998637 7933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.001536 7933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:12.001739 7933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.007959 7933 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41413
I20260807 15:02:12.008011 8001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41413 every 8 connection(s)
I20260807 15:02:12.008541 7933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:12.010056 8002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:12.011677 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 7933
I20260807 15:02:12.011778 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:12.012007 1535 external_mini_cluster.cc:1428] Setting key eece35a72b904fa12178eff387ab1146
I20260807 15:02:12.013202 8002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Bootstrap starting.
I20260807 15:02:12.013955 8002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:12.014461 8002 log.cc:824] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:12.015467 8002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: No bootstrap required, opened a new log
I20260807 15:02:12.018282 8002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } }
I20260807 15:02:12.018512 8002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:12.018558 8002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649e709be6fc4b46b52150eb1b3f2e6c, State: Initialized, Role: FOLLOWER
I20260807 15:02:12.018782 8002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } }
I20260807 15:02:12.018904 8002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:12.018944 8002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:12.018994 8002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:12.019925 8002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } }
I20260807 15:02:12.020025 8002 leader_election.cc:302] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 649e709be6fc4b46b52150eb1b3f2e6c; no voters:
I20260807 15:02:12.020247 8002 leader_election.cc:288] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:12.020310 8007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 FOLLOWER]: Leader election won for term 1
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:12.020532 8007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 LEADER]: Becoming Leader. State: Replica: 649e709be6fc4b46b52150eb1b3f2e6c, State: Running, Role: LEADER
I20260807 15:02:12.020562 8002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:12.020630 8007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } }
I20260807 15:02:12.021142 8010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } }
I20260807 15:02:12.021446 8010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:12.021361 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 649e709be6fc4b46b52150eb1b3f2e6c. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } }
I20260807 15:02:12.021728 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:12.021731 8017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:12.023233 8017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:12.024950 8017 catalog_manager.cc:1347] Generated new cluster ID: de00166f4a8940c38f29adc42af1d309
I20260807 15:02:12.025004 8017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:12.025631 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.012813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.1.127.254:41413)
0807 15:02:12.013007 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.013020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.013048 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:02:12.013559 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.013568 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.013631 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.013766 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.015000 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.016047 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.017302 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.017751 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.017845 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.021174 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.021179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.021180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.021194 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.023539 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.024011 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.024015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.024015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.024125 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.024366 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.024367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.024367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.024826 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.025001 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.025272 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.025419 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260807 15:02:12.034523 8017 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:12.035398 8017 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:12.039412 8017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Generated new TSK 0
I20260807 15:02:12.039714 8017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
I20260807 15:02:12.074883 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:12.168746 8029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:12.169036 8029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:12.169094 8029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:12.170441 8029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:12.170522 8029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:12.170572 8029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:12.172269 8029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:12.172600 8029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:12.172866 8029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:12.178598 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.178598 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.178743 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.178857 8029 server_base.cc:1034] running on GCE node
I20260807 15:02:12.179260 8029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:12.179600 8029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:12.180822 8029 hybrid_clock.cc:648] HybridClock initialized: now 1786114932180805 us; error 35 us; skew 500 ppm
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:12.184422 8029 init.cc:377] Logged in from keytab as oryx/127.1.127.193@KRBTEST.COM (short username oryx)
I20260807 15:02:12.185001 8029 webserver.cc:460] Webserver started at http://127.1.127.193:40657/ using document root <none> and password file <none>
I20260807 15:02:12.185256 8029 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.185309 8029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.185451 8029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:12.186604 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "6a766c4220014ba796fa0a75c7b08f53"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "d7a1004054466c7e0a75c6c34a859341"
server_key_iv: "3d71de9fe9351493d8bcee063d711167"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.187055 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "6a766c4220014ba796fa0a75c7b08f53"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "d7a1004054466c7e0a75c6c34a859341"
server_key_iv: "3d71de9fe9351493d8bcee063d711167"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.188563 8029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:12.189631 8045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.189888 8029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:12.190016 8029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "6a766c4220014ba796fa0a75c7b08f53"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "d7a1004054466c7e0a75c6c34a859341"
server_key_iv: "3d71de9fe9351493d8bcee063d711167"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.190114 8029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.207779 8029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.210466 8029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:12.210824 8029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.211163 8029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:12.211551 8029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:12.211608 8029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.211647 8029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:12.211688 8029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.220016 8029 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35511
I20260807 15:02:12.220057 8164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35511 every 8 connection(s)
I20260807 15:02:12.220492 8029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:12.222111 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8029
I20260807 15:02:12.222236 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:12.222492 1535 external_mini_cluster.cc:1428] Setting key fd8b2a6a7e6c4654205fece960afb96b
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:02:12.233788 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.221366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:44951 (local address 127.1.127.254:41413)
0807 15:02:12.221432 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.221434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.221672 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:02:12.222591 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.222593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.222610 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.222660 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.224272 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.224804 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.226044 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.226262 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.226355 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.229573 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.229576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.229577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.229588 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.231757 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.232397 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.232399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.232399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.232475 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.232891 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.232893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.232893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.233042 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.233140 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.233482 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.233646 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:02:12.233933 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.221263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:44951)
0807 15:02:12.221530 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.221576 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.222438 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.222907 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.222912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.223795 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.224122 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.224130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.224917 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.224918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.225908 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.225911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.226371 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.226372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.226509 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.227334 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.227345 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.229427 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.231930 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.231998 (+ 68us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.232001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.232302 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.232590 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.232592 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.232592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.232788 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.233131 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.233140 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.233322 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.233448 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.233663 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
I20260807 15:02:12.234760 8165 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:12.234884 8165 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:12.235096 8165 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:12.235759 7966 ts_manager.cc:194] Registered new tserver with Master: 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.194@KRBTEST.COM" created.
I20260807 15:02:12.236438 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:44951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
I20260807 15:02:12.258742 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:12.354625 8170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:12.354946 8170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:12.354980 8170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:12.356559 8170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:12.356637 8170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:12.356673 8170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:12.358532 8170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:12.358861 8170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:12.359089 8170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:12.364956 8175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.365051 8176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.364956 8178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.365139 8170 server_base.cc:1034] running on GCE node
I20260807 15:02:12.365440 8170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:12.365712 8170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:12.366873 8170 hybrid_clock.cc:648] HybridClock initialized: now 1786114932366853 us; error 35 us; skew 500 ppm
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:12.370261 8170 init.cc:377] Logged in from keytab as oryx/127.1.127.194@KRBTEST.COM (short username oryx)
I20260807 15:02:12.370985 8170 webserver.cc:460] Webserver started at http://127.1.127.194:45281/ using document root <none> and password file <none>
I20260807 15:02:12.371249 8170 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.371321 8170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.371680 8170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:12.372967 8170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "ca89db920c514d6794bfdec0c0b56014"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "1f5533d0d72de47e6de90219c9a62767"
server_key_iv: "88d67439a7aef60ba1ce7fdeac0a372f"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.373350 8170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "ca89db920c514d6794bfdec0c0b56014"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "1f5533d0d72de47e6de90219c9a62767"
server_key_iv: "88d67439a7aef60ba1ce7fdeac0a372f"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.375026 8170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:02:12.375917 8186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.376151 8170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:12.376262 8170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "ca89db920c514d6794bfdec0c0b56014"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "1f5533d0d72de47e6de90219c9a62767"
server_key_iv: "88d67439a7aef60ba1ce7fdeac0a372f"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.376339 8170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.390194 8170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.392743 8170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:12.392937 8170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.393260 8170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:12.393667 8170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:12.393724 8170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.393760 8170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:12.393786 8170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.400395 8170 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:37837
I20260807 15:02:12.400492 8304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:37837 every 8 connection(s)
I20260807 15:02:12.400853 8170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:12.405858 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8170
I20260807 15:02:12.405953 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:12.406184 1535 external_mini_cluster.cc:1428] Setting key 357f19fafd07ce5447c32833e38c0d4d
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:12.414103 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.401818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:48259 (local address 127.1.127.254:41413)
0807 15:02:12.402028 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.402031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.402238 (+ 207us) server_negotiation.cc:407] Connection header received
0807 15:02:12.403101 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.403103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.403118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.403183 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.404530 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.405126 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.406586 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.406946 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.407037 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.409907 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.409910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.409910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.409921 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.411919 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.412806 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.412808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.412808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.412883 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.413272 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.413274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.413274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.413442 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.413531 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.413886 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.413993 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 15:02:12.414414 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.401739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:48259)
0807 15:02:12.402079 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.402146 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.402896 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.403311 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.403316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.404147 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.404385 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.404391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.405239 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.405240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.406465 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.406469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.407080 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.407082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.407224 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.407881 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.407891 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.409769 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.412227 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.412240 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.412244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.412713 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.412971 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.412972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.412973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.413179 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.413523 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.413532 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.413715 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.413843 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.414061 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
I20260807 15:02:12.414997 8305 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:12.415103 8305 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:12.415318 8305 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:12.415803 7966 ts_manager.cc:194] Registered new tserver with Master: ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:12.416251 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:48259
WARNING: no policy specified for oryx/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
I20260807 15:02:12.442883 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:12.530637 8310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:12.530967 8310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:12.531000 8310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:12.532344 8310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:12.532402 8310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:12.532433 8310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:12.534117 8310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:12.534428 8310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:12.534739 8310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:12.540766 8316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.540772 8318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.540786 8315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.541065 8310 server_base.cc:1034] running on GCE node
I20260807 15:02:12.541329 8310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:12.541652 8310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:12.542842 8310 hybrid_clock.cc:648] HybridClock initialized: now 1786114932542831 us; error 44 us; skew 500 ppm
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:12.545764 8310 init.cc:377] Logged in from keytab as oryx/127.1.127.195@KRBTEST.COM (short username oryx)
I20260807 15:02:12.546335 8310 webserver.cc:460] Webserver started at http://127.1.127.195:41699/ using document root <none> and password file <none>
I20260807 15:02:12.546550 8310 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.546594 8310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.546759 8310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:12.547912 8310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "6cf09af3607c4e79a64d456d6d496c97"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "f031089d681b7827f0e42a5ebbd93ab2"
server_key_iv: "ffaa7df0f51abf69f9502234ea800b3c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.548256 8310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "6cf09af3607c4e79a64d456d6d496c97"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "f031089d681b7827f0e42a5ebbd93ab2"
server_key_iv: "ffaa7df0f51abf69f9502234ea800b3c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.549706 8310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:12.550638 8326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.550913 8310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:12.551034 8310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6cf09af3607c4e79a64d456d6d496c97"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "f031089d681b7827f0e42a5ebbd93ab2"
server_key_iv: "ffaa7df0f51abf69f9502234ea800b3c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.551138 8310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.567463 8310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.569985 8310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:12.570163 8310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.570439 8310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:12.570851 8310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:12.570907 8310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.570953 8310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:12.570981 8310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.577913 8310 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38737
I20260807 15:02:12.578069 8447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38737 every 8 connection(s)
I20260807 15:02:12.578392 8310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:12.579566 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8310
I20260807 15:02:12.579682 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:12.579926 1535 external_mini_cluster.cc:1428] Setting key da1b22b74231520ddace007491f31098
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:12.590072 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.579376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46135 (local address 127.1.127.254:41413)
0807 15:02:12.579451 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.579453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.579815 (+ 362us) server_negotiation.cc:407] Connection header received
0807 15:02:12.580676 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.580676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.580689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.580741 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.582164 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.582749 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.583860 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.584116 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.584175 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.586752 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.586754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.586755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.586764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.588378 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.588891 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.588892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.588892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.588962 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.589347 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.589348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.589348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.589480 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.589559 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.589868 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.589960 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:02:12.590317 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.579424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:46135)
0807 15:02:12.579650 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.579711 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.580548 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.580825 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.580831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.581719 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.582020 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.582027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.582838 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.582838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.583745 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.583748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.584202 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.584203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.584322 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.584892 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.584901 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.586606 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.588492 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.588541 (+ 49us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.588543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.588801 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.589119 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.589119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.589120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.589268 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.589561 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.589570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.589723 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.589833 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.590027 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
I20260807 15:02:12.590932 8448 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:12.591133 8448 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:12.591323 8448 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:12.591792 7966 ts_manager.cc:194] Registered new tserver with Master: 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737)
I20260807 15:02:12.592463 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:46135
I20260807 15:02:12.592540 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.127.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.253.keytab.
Entry for principal oryx/127.1.127.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.253.keytab.
I20260807 15:02:12.628290 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:38271
--webserver_interface=127.1.127.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.253.keytab
--principal=oryx/127.1.127.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41413,127.1.127.253:38271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:12.723263 8453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:12.723500 8453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:12.723524 8453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:12.724773 8453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:12.724820 8453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:12.724835 8453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:12.724848 8453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:12.724861 8453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:12.726403 8453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41413,127.1.127.253:38271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.253.keytab
--principal=oryx/127.1.127.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:38271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.127.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
W20260807 15:02:12.726526 8453 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.127.254:41413,127.1.127.253:38271'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 15:02:12.726653 8453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:12.726936 8453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:12.732699 8459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.732851 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.732728 8458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.732717 8453 server_base.cc:1034] running on GCE node
I20260807 15:02:12.733364 8453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:12.733763 8453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:12.734979 8453 hybrid_clock.cc:648] HybridClock initialized: now 1786114932734954 us; error 41 us; skew 500 ppm
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:12.738303 8453 init.cc:377] Logged in from keytab as oryx/127.1.127.253@KRBTEST.COM (short username oryx)
I20260807 15:02:12.738988 8453 webserver.cc:460] Webserver started at http://127.1.127.253:33633/ using document root <none> and password file <none>
I20260807 15:02:12.739256 8453 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.739331 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.739506 8453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:12.740768 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/instance:
uuid: "5eb970024e7249fe80916cba830ee843"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "fa0b65d2d0e37dd573167a5d254e6d78"
server_key_iv: "e9258e21a91a68f16c07a05f3d67ba0c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.741209 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal/instance:
uuid: "5eb970024e7249fe80916cba830ee843"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "fa0b65d2d0e37dd573167a5d254e6d78"
server_key_iv: "e9258e21a91a68f16c07a05f3d67ba0c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.742890 8453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:02:12.743899 8469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.744143 8453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:12.744251 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "5eb970024e7249fe80916cba830ee843"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "fa0b65d2d0e37dd573167a5d254e6d78"
server_key_iv: "e9258e21a91a68f16c07a05f3d67ba0c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.744328 8453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.756767 8453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.759675 8453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:12.759938 8453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.764348 8453 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.253:38271
I20260807 15:02:12.764405 8523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.253:38271 every 8 connection(s)
I20260807 15:02:12.764791 8453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
I20260807 15:02:12.765173 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8453
I20260807 15:02:12.766213 8524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:12.768409 8524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.253@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:12.777359 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.765985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.1.127.253:38271)
0807 15:02:12.766164 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.766173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.766202 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:02:12.766799 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.766807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.766868 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.766988 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.767799 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.768840 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.769952 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.770522 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.770635 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.773807 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.773812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.773813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.773826 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.775745 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.776134 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.776137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.776138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.776225 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.776473 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.776474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.776474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.776723 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.776873 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.777000 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.777142 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260807 15:02:12.778625 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.769213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.1.127.254:41413)
0807 15:02:12.769269 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.769271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.769448 (+ 177us) server_negotiation.cc:407] Connection header received
0807 15:02:12.769520 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.769522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.769533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.769595 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.770149 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.770965 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.772566 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.772960 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.773052 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.775709 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.775710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.775711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.775723 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.777285 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.777686 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.777687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.777688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.777752 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.777945 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.777946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.777947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.778163 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.778249 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.778439 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.778532 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:02:12.778638 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.769109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52630)
0807 15:02:12.769301 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.769348 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.769460 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.769610 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.769612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.769977 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.770067 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.770069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.771162 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.771164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.772435 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.772439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.773077 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.773078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.773153 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.773818 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.773825 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.775612 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.777366 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.777371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.777373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.777601 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.777817 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.777817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.777818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.777882 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.778239 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.778245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.778297 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.778406 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.778527 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":115,"threads_started":1}
I20260807 15:02:12.779309 8524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.253@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
I20260807 15:02:12.786885 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.779635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46240)
0807 15:02:12.779733 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.779748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.779783 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.779883 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.779884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.780055 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.780089 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.780091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.780840 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.780841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.781325 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.781326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.781748 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.781750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.781800 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.782242 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.782247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.783704 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.785439 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.785440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.785440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.785626 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.786107 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.786108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.786108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.786180 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.786512 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.786513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.786559 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.786625 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.786798 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 15:02:12.786983 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.779691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.1.127.253:38271)
0807 15:02:12.779761 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.779763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.779775 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:12.779822 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.779822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.779831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.779877 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.780166 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.780670 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.781441 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.781658 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.781723 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.783824 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.783825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.783826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.783832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.785311 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.785718 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.785719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.785720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.785795 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.786283 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.786284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.786284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.786423 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.786489 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.786768 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.786862 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":49}
I20260807 15:02:12.788921 8524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Bootstrap starting.
I20260807 15:02:12.789891 8524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:12.790254 8524 log.cc:824] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:12.791163 8524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: No bootstrap required, opened a new log
I20260807 15:02:12.792590 8524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } }
I20260807 15:02:12.792737 8524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:12.792760 8524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb970024e7249fe80916cba830ee843, State: Initialized, Role: FOLLOWER
I20260807 15:02:12.792891 8524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } }
I20260807 15:02:12.793202 8524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:12.793197 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } } }
I20260807 15:02:12.793308 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:02:12.795224 8541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:02:12.795298 8541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:02:12.797304 8453 master_runner.cc:428] Detected that this master 5eb970024e7249fe80916cba830ee843 is joining an existing cluster
I20260807 15:02:12.797434 8453 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.127.253:38271
I20260807 15:02:12.806051 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.799180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.1.127.254:41413)
0807 15:02:12.799424 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.799426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.799435 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:12.799461 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.799462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.799473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.799534 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.799923 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.800788 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.801725 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.802041 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.802123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.803118 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.803119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.803120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.803127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.804802 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805254 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.805255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.805256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.805317 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805516 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.805517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.805517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.805632 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.805707 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.805858 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.805940 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260807 15:02:12.806064 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.798485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52642)
0807 15:02:12.798655 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.798668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.798751 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.799560 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.799560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.799768 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.799823 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.799825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.800892 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.800893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.801632 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.801634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.802121 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.802122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.802187 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.802561 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.802565 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.802995 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.804912 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.804913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.804913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.805181 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.805385 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.805386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.805447 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.805698 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.805699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.805745 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.805830 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.805941 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":3,"thread_start_us":546,"threads_started":1}
I20260807 15:02:12.807029 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.798486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46248)
0807 15:02:12.799136 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.799149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.799222 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.799394 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.799395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.799686 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.799791 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.799794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.800512 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.800513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.801245 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.801247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.801783 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.801784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.801843 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.802400 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.802404 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.802958 (+ 554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.805402 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.805403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.805625 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.805832 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.805833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.805891 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.806552 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.806553 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.806602 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.806719 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.806842 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":110,"threads_started":1}
I20260807 15:02:12.807037 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.798663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.1.127.253:38271)
0807 15:02:12.798750 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.798752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.799238 (+ 486us) server_negotiation.cc:407] Connection header received
0807 15:02:12.799272 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.799273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.799285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.799361 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.799963 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.800426 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.801343 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.801685 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.801776 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.803074 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.803075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.803076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.803083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.805305 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805710 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.805710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.805711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.805764 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.805976 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.805976 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.805977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.806107 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.806162 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.806741 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.806915 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":72}
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.254@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
I20260807 15:02:12.818318 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.809035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.1.127.253:38271)
0807 15:02:12.809176 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.809179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.809242 (+ 63us) server_negotiation.cc:407] Connection header received
0807 15:02:12.809330 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.809330 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.809343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.809416 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.809871 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.810734 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.811579 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.811992 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.812092 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.815219 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.815221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.815221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.815232 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.816879 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.817331 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.817332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.817332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.817399 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.817661 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.817662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.817662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.817814 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.817887 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.818125 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.818212 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 15:02:12.818406 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.808969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46252)
0807 15:02:12.809136 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.809174 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.809246 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.809420 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.809422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.809730 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.809791 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.809793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.810820 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.810821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.811487 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.811491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.812116 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.812117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.812198 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.813272 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.813277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.815107 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.817003 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.817013 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.817015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.817254 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.817505 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.817506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.817506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.817590 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.817921 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.817929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.817993 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.818084 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.818276 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260807 15:02:12.819309 7966 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.127.253:38271: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5eb970024e7249fe80916cba830ee843"
member_type: NON_VOTER
last_known_addr {
host: "127.1.127.253"
port: 38271
}
attrs {
promote: true
}
}
dest_uuid: "649e709be6fc4b46b52150eb1b3f2e6c"
cas_config_opid_index: -1
I20260807 15:02:12.819779 7966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } }
I20260807 15:02:12.820480 8551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5eb970024e7249fe80916cba830ee843 (127.1.127.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } }
I20260807 15:02:12.820832 8497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:12.821554 8551 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.127.253:38271
I20260807 15:02:12.821853 8497 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 15:02:12.821909 8453 master.cc:537] Master@127.1.127.253:38271 shutting down...
I20260807 15:02:12.821858 8550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } }
I20260807 15:02:12.822409 8551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:12.821959 8550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
W20260807 15:02:12.823345 7951 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 15:02:12.824357 7951 consensus_peers.cc:487] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c -> Peer 5eb970024e7249fe80916cba830ee843 (127.1.127.253:38271): Couldn't send request to peer 5eb970024e7249fe80916cba830ee843. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:02:12.824995 8453 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:12.825110 8453 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:12.825169 8453 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: stopping tablet replica
I20260807 15:02:12.838313 8453 master.cc:552] Master@127.1.127.253:38271 shutdown complete.
I20260807 15:02:12.838408 8453 master_runner.cc:305] Clearing existing system tablet
I20260807 15:02:12.838544 8453 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.838613 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.839363 8453 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.839874 8556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.840061 8453 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.840122 8453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.840147 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "5eb970024e7249fe80916cba830ee843"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "fa0b65d2d0e37dd573167a5d254e6d78"
server_key_iv: "e9258e21a91a68f16c07a05f3d67ba0c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.841303 8453 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:12.842941 8453 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:02:12.843026 8453 log.cc:1196] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 15:02:12.843259 8453 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Deleting consensus metadata
I20260807 15:02:12.843402 8453 master_runner.cc:315] Copying system tablet from 127.1.127.254:41413
I20260807 15:02:12.844061 8453 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: tablet copy: Beginning tablet copy session from remote peer at address 127.1.127.254:41413
I20260807 15:02:12.851545 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.844459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52654)
0807 15:02:12.844597 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:12.844612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:12.844687 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:12.844835 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:12.844835 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:12.845000 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:12.845067 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.845069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.846059 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.846060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.846643 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:12.846645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.847248 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.847249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:12.847316 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.848015 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:12.848021 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:12.848344 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:12.850404 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.850405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.850406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.850603 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.850841 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:12.850841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:12.850842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:12.850911 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:12.851195 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:12.851196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:12.851241 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:02:12.851310 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:02:12.851465 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260807 15:02:12.851560 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.844580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.1.127.254:41413)
0807 15:02:12.844658 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.844660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.844673 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:02:12.844739 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.844740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.844758 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.844830 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.845143 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.845948 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.846813 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.847149 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.847231 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.848450 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.848451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.848452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.848460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.850314 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.850701 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.850702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.850703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.850773 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.850975 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.850976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.850976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.851121 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.851198 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.851358 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.851449 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:02:12.852023 7986 tablet_copy_service.cc:140] P 649e709be6fc4b46b52150eb1b3f2e6c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5eb970024e7249fe80916cba830ee843 ({username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:52654)
I20260807 15:02:12.852187 7986 tablet_copy_service.cc:161] P 649e709be6fc4b46b52150eb1b3f2e6c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5eb970024e7249fe80916cba830ee843 at {username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:52654: session id = 5eb970024e7249fe80916cba830ee843-00000000000000000000000000000000
I20260807 15:02:12.853010 7986 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:02:12.854023 8453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:02:12.855475 8453 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: tablet copy: Starting download of 0 data blocks...
I20260807 15:02:12.855656 8453 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: tablet copy: Starting download of 1 WAL segments...
I20260807 15:02:12.856775 8453 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:02:12.857712 7986 tablet_copy_service.cc:342] P 649e709be6fc4b46b52150eb1b3f2e6c: Request end of tablet copy session 5eb970024e7249fe80916cba830ee843-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:52654
I20260807 15:02:12.857784 7986 tablet_copy_service.cc:433] P 649e709be6fc4b46b52150eb1b3f2e6c: ending tablet copy session 5eb970024e7249fe80916cba830ee843-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5eb970024e7249fe80916cba830ee843
I20260807 15:02:12.861271 8453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:12.862820 8566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:12.862828 8563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.862964 8453 server_base.cc:1034] running on GCE node
W20260807 15:02:12.871975 8564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:12.872293 8453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:12.872529 8453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:12.873689 8453 hybrid_clock.cc:648] HybridClock initialized: now 1786114932873683 us; error 35 us; skew 500 ppm
Aug 07 15:02:12 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114932, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:12.875406 8453 init.cc:377] Logged in from keytab as oryx/127.1.127.253@KRBTEST.COM (short username oryx)
I20260807 15:02:12.875751 8453 webserver.cc:460] Webserver started at http://127.1.127.253:41241/ using document root <none> and password file <none>
I20260807 15:02:12.875936 8453 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:12.875982 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:12.876685 8453 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:02:12.877244 8573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.877437 8453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:12.877557 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "5eb970024e7249fe80916cba830ee843"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "fa0b65d2d0e37dd573167a5d254e6d78"
server_key_iv: "e9258e21a91a68f16c07a05f3d67ba0c"
server_key_version: "encryptionkey@0"
I20260807 15:02:12.877633 8453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:12.890156 8453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:12.892177 8453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:12.895711 8453 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.253:38271
I20260807 15:02:12.896075 8627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.253:38271 every 8 connection(s)
I20260807 15:02:12.896152 8453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
I20260807 15:02:12.897011 8628 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:02:12.897529 8628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Bootstrap starting.
I20260807 15:02:12.900429 8628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:02:12.900671 8628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Bootstrap complete.
I20260807 15:02:12.900980 8628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } }
I20260807 15:02:12.901077 8628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5eb970024e7249fe80916cba830ee843, State: Initialized, Role: LEARNER
I20260807 15:02:12.901124 8628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } }
I20260807 15:02:12.901342 8628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:12.901314 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } }
I20260807 15:02:12.901388 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:02:12.902941 8642 catalog_manager.cc:1259] Loaded cluster ID: de00166f4a8940c38f29adc42af1d309
I20260807 15:02:12.902997 8642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: loading cluster ID for follower catalog manager: success
I20260807 15:02:12.904049 8642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: acquiring CA information for follower catalog manager: success
I20260807 15:02:12.904341 8642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:02:12.937078 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:12.929892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.1.127.253:38271)
0807 15:02:12.930097 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:12.930099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:12.930105 (+ 6us) server_negotiation.cc:407] Connection header received
0807 15:02:12.930129 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:12.930130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:12.930140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:12.930214 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:12.930882 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.931617 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.932523 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:12.932748 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:12.932826 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:12.934126 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:12.934128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:12.934128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:12.934135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:12.935687 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.936072 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.936073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.936073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.936134 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:12.936355 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:12.936356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:12.936356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:12.936480 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:12.936622 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:12.936785 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:02:12.936884 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.1.127.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.252.keytab.
Entry for principal oryx/127.1.127.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.252.keytab.
I20260807 15:02:12.973510 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:40673
--webserver_interface=127.1.127.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.252.keytab
--principal=oryx/127.1.127.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:13.061331 8646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:13.061610 8646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:13.061651 8646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:13.062922 8646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:13.062992 8646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:13.063015 8646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:13.063036 8646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:13.063071 8646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:13.064680 8646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.252.keytab
--principal=oryx/127.1.127.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:40673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.127.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:13.064990 8646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:13.065243 8646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:13.070652 8654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.070637 8651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.070637 8652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.071338 8646 server_base.cc:1034] running on GCE node
I20260807 15:02:13.071660 8646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:13.072027 8646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:13.073242 8646 hybrid_clock.cc:648] HybridClock initialized: now 1786114933073220 us; error 43 us; skew 500 ppm
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:13.076468 8646 init.cc:377] Logged in from keytab as oryx/127.1.127.252@KRBTEST.COM (short username oryx)
I20260807 15:02:13.077100 8646 webserver.cc:460] Webserver started at http://127.1.127.252:36885/ using document root <none> and password file <none>
I20260807 15:02:13.077359 8646 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.077427 8646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.077579 8646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:13.078872 8646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/instance:
uuid: "a990961ebcc34988835efae4d61865b9"
format_stamp: "Formatted at 2026-08-07 15:02:13 on dist-test-slave-8tdl"
server_key: "181f68581da6554dc3fe205dbcfc890f"
server_key_iv: "fd858b05888d45f81a22f1bb330af5dd"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.079270 8646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal/instance:
uuid: "a990961ebcc34988835efae4d61865b9"
format_stamp: "Formatted at 2026-08-07 15:02:13 on dist-test-slave-8tdl"
server_key: "181f68581da6554dc3fe205dbcfc890f"
server_key_iv: "fd858b05888d45f81a22f1bb330af5dd"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.080886 8646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:13.081789 8661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.082017 8646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:13.082129 8646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "a990961ebcc34988835efae4d61865b9"
format_stamp: "Formatted at 2026-08-07 15:02:13 on dist-test-slave-8tdl"
server_key: "181f68581da6554dc3fe205dbcfc890f"
server_key_iv: "fd858b05888d45f81a22f1bb330af5dd"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.082206 8646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.094838 8646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:13.097610 8646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:13.097838 8646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:13.103120 8646 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.252:40673
I20260807 15:02:13.103166 8716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.252:40673 every 8 connection(s)
I20260807 15:02:13.103610 8646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
I20260807 15:02:13.104772 8717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:13.106546 8717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:02:13.110033 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8646
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.252@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
I20260807 15:02:13.118113 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.107596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.1.127.254:41413)
0807 15:02:13.107648 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.107650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.107810 (+ 160us) server_negotiation.cc:407] Connection header received
0807 15:02:13.108305 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.108307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.108318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.108388 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.109072 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.109622 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.110964 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.111345 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.111417 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.114327 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.114329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.114329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.114337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.116418 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.117025 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.117027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.117027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.117100 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.117372 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.117373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.117373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.117506 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.117580 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.117921 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.118014 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:02:13.118317 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.107365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52660)
0807 15:02:13.107603 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.107650 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.108190 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.108508 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.108512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.108771 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.108982 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.108987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.109712 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.109713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.110871 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.110875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.111509 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.111510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.111624 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.112229 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.112240 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.114208 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.116532 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.116540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.116542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.116800 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.117170 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.117171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.117172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.117299 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.117580 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.117588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.117740 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.117896 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.118074 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":154,"threads_started":1}
I20260807 15:02:13.119035 8717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:02:13.121671 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.110818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.1.127.252:40673)
0807 15:02:13.111040 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.111045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.111064 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:02:13.111084 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.111086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.111107 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.111307 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.112097 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.113222 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.114113 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.114491 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.114546 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.117639 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.117648 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.117648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.117656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.120085 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.120545 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.120547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.120548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.120623 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.120846 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.120847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.120847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.121019 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.121135 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.121373 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.121494 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":126,"threads_started":1}
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.252@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
I20260807 15:02:13.127250 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.119454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.1.127.253:38271)
0807 15:02:13.119527 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.119529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.119539 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:13.119568 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.119569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.119583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.119642 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.119988 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.120704 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.121831 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.122170 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.122211 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.124352 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.124354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.124355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.124364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.125987 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.126361 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.126362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.126362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.126427 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.126635 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.126636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.126636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.126819 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.126897 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.127054 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.127147 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:02:13.127250 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.119369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46302)
0807 15:02:13.119411 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.119452 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.119509 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.119646 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.119647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.119784 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.119839 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.119841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.120835 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.120836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.121699 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.121702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.122311 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.122312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.122407 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.122882 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.122886 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.124238 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.126093 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.126095 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.126095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.126284 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.126502 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.126503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.126503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.126565 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.126887 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.126888 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.126940 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.127028 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.127144 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 15:02:13.127826 8717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.252@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
I20260807 15:02:13.135293 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.128234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.1.127.252:40673)
0807 15:02:13.128288 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.128290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.128319 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:02:13.128338 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.128339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.128349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.128398 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.128704 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.129281 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.129928 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.130211 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.130276 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.132374 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.132375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.132376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.132383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.133950 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.134337 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.134338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.134338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.134396 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.134628 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.134629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.134629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.134841 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.134902 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.135089 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.135223 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":35}
I20260807 15:02:13.135398 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.128185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60088)
0807 15:02:13.128234 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.128246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.128285 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.128403 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.128404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.128589 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.128629 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.128631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.129371 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.129372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.129850 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.129852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.130325 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.130326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.130423 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.130953 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.130957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.132287 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.134070 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.134072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.134072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.134253 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.134486 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.134487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.134488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.134552 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.134933 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.134934 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.134987 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.135058 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.135154 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:02:13.137059 8717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Bootstrap starting.
I20260807 15:02:13.138011 8717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:13.138350 8717 log.cc:824] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:13.139325 8717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: No bootstrap required, opened a new log
I20260807 15:02:13.141000 8717 raft_consensus.cc:348] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } }
I20260807 15:02:13.141172 8717 raft_consensus.cc:374] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:13.141222 8717 raft_consensus.cc:729] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a990961ebcc34988835efae4d61865b9, State: Initialized, Role: FOLLOWER
I20260807 15:02:13.141386 8717 consensus_queue.cc:260] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } }
I20260807 15:02:13.141726 8717 sys_catalog.cc:564] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:13.141711 8722 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } } }
I20260807 15:02:13.141834 8722 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:02:13.144040 8734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:02:13.144136 8734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:02:13.147322 8646 master_runner.cc:428] Detected that this master a990961ebcc34988835efae4d61865b9 is joining an existing cluster
I20260807 15:02:13.147395 8646 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.127.252:40673
I20260807 15:02:13.156584 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.148793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.1.127.253:38271)
0807 15:02:13.148847 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.148849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.149053 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:02:13.149175 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.149176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.149190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.149226 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.149796 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.150612 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.151893 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152195 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152240 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.153619 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.153622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.153622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.153630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.155320 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.155732 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.155733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.155734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.155806 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.156004 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.156004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.156005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.156136 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.156212 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.156386 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.156478 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:02:13.156602 8740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.148762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46308)
0807 15:02:13.148928 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.148939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.149087 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.149312 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.149313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.149581 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.149697 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.149699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.150742 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.150743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.151810 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.151811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152329 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.152414 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.152975 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.152980 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.153377 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.155415 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.155416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.155417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.155649 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.155871 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.155872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.155872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.155940 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.156201 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.156202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.156270 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.156347 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.156484 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":45,"thread_start_us":108,"threads_started":1}
I20260807 15:02:13.173923 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8029
I20260807 15:02:13.175135 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.149167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.1.127.252:40673)
0807 15:02:13.149205 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.149207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.149387 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:02:13.149470 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.149470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.149478 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.149504 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.149955 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.150647 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.151787 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152078 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152116 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.172100 (+ 19984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.172102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.172102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.172115 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.173896 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.174304 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.174305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.174306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.174356 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.174530 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.174531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.174531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.174661 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.174743 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.174916 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.174996 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:02:13.175433 8742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.149024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60092)
0807 15:02:13.149247 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.149256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.149408 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.149556 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.149557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.149816 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.149878 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.149879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.150764 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.150765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.151693 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.151696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152535 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.152603 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.171428 (+ 18825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.171434 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.171868 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.173981 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.173982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.173982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.174227 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.174416 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.174417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.174417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.174471 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.174780 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.174781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.174826 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.175249 (+ 423us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.175337 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":34,"thread_start_us":71,"threads_started":1}
I20260807 15:02:13.179562 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.148797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.1.127.254:41413)
0807 15:02:13.148848 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.148850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.149319 (+ 469us) server_negotiation.cc:407] Connection header received
0807 15:02:13.149447 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.149448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.149461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.149494 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.150069 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.150900 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.151990 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152295 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152351 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.172127 (+ 19776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.172129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.172129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.172140 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.174180 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.178491 (+ 4311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.178493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.178493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.178585 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.178882 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.178883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.178883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.179054 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.179103 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.179348 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.179460 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:02:13.180317 8741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.148914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52670)
0807 15:02:13.149063 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.149072 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.149352 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.149566 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.149567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.149882 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.149963 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.149965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.151006 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.151007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.151878 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.151881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.152446 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.152447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.152512 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.171370 (+ 18858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.171378 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.171972 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.178076 (+ 6104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.178078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.178078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.178346 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.178738 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.178739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.178739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.178805 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.179167 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.179168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.179215 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.180046 (+ 831us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.180206 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260807 15:02:13.187115 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:35511
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:41413
--webserver_port=40657
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114931, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.254@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
I20260807 15:02:13.192890 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.184548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60098)
0807 15:02:13.184665 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.184677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.184717 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.184895 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.184896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.185118 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.185179 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.185182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.186118 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.186120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.187005 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.187008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.187528 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.187530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.187609 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.188078 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.188085 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.189572 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.191614 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.191615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.191616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.191863 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.192108 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.192109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.192109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.192189 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.192522 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.192523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.192574 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.192637 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.192781 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 15:02:13.193291 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.184683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.1.127.252:40673)
0807 15:02:13.184744 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.184746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.184755 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:13.184794 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.184794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.184825 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.184889 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.185263 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.186019 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.187101 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.187393 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.187466 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.189693 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.189695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.189695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.189703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.191506 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.191954 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.191956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.191956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.192029 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.192272 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.192273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.192274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.192431 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.192493 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.193114 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.193202 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:02:13.193945 7966 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.127.252:40673: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a990961ebcc34988835efae4d61865b9"
member_type: NON_VOTER
last_known_addr {
host: "127.1.127.252"
port: 40673
}
attrs {
promote: true
}
}
dest_uuid: "649e709be6fc4b46b52150eb1b3f2e6c"
cas_config_opid_index: 5
I20260807 15:02:13.194276 7966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } }
I20260807 15:02:13.194767 8551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a990961ebcc34988835efae4d61865b9 (127.1.127.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.195294 8689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:13.196135 8551 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.127.252:40673
I20260807 15:02:13.196220 8551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.196295 8551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.196738 8646 master.cc:537] Master@127.1.127.252:40673 shutting down...
I20260807 15:02:13.196998 8689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:02:13.197324 8550 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:02:13.198173 7951 consensus_peers.cc:487] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c -> Peer a990961ebcc34988835efae4d61865b9 (127.1.127.252:40673): Couldn't send request to peer a990961ebcc34988835efae4d61865b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:02:13.200390 8646 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:13.200507 8646 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:13.200575 8646 tablet_replica.cc:330] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: stopping tablet replica
I20260807 15:02:13.202368 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.195845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.1.127.253:38271)
0807 15:02:13.195898 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.195900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.197789 (+ 1889us) server_negotiation.cc:407] Connection header received
0807 15:02:13.197903 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.197904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.197932 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.197974 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:13.198215 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.199013 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.200874 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.201508 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.201559 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.201632 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.202243 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.202292 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:02:13.202795 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.194919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:46316)
0807 15:02:13.197715 (+ 2796us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.197731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.197823 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.198071 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.198072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.198074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:13.198126 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.198129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.199210 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.199211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.200761 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.200765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.201891 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.201892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.201966 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.201999 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.202646 (+ 647us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.202721 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2770}
I20260807 15:02:13.203323 8602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 LEARNER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:02:13.203627 8550 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:02:13.204241 8602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a990961ebcc34988835efae4d61865b9 (127.1.127.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.204808 8646 master.cc:552] Master@127.1.127.252:40673 shutdown complete.
I20260807 15:02:13.204846 8646 master_runner.cc:305] Clearing existing system tablet
I20260807 15:02:13.204979 8646 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.205021 8646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.205385 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 649e709be6fc4b46b52150eb1b3f2e6c. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.205468 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:02:13.205497 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: NON_VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: true } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.205528 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:02:13.205747 8646 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:02:13.206167 8550 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: attempting to promote NON_VOTER 5eb970024e7249fe80916cba830ee843 to VOTER
I20260807 15:02:13.206199 8747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.206283 8646 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.206317 8646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.206346 8646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "a990961ebcc34988835efae4d61865b9"
format_stamp: "Formatted at 2026-08-07 15:02:13 on dist-test-slave-8tdl"
server_key: "181f68581da6554dc3fe205dbcfc890f"
server_key_iv: "fd858b05888d45f81a22f1bb330af5dd"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.206385 8550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } }
I20260807 15:02:13.207291 8646 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:13.207381 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.207230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60102)
0807 15:02:13.207303 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.207350 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 15:02:13.207554 8602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 LEARNER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 15:02:13.207918 8551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:02:13.208282 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.208169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60116)
0807 15:02:13.208233 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.208257 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 15:02:13.208587 7951 consensus_peers.cc:487] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c -> Peer a990961ebcc34988835efae4d61865b9 (127.1.127.252:40673): Couldn't send request to peer a990961ebcc34988835efae4d61865b9. Status: Network error: Client connection negotiation failed: client connection to 127.1.127.252:40673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:02:13.208832 8646 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:02:13.208907 8646 log.cc:1196] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 15:02:13.209018 8551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5eb970024e7249fe80916cba830ee843 (127.1.127.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.209132 8646 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Deleting consensus metadata
I20260807 15:02:13.209297 8646 master_runner.cc:315] Copying system tablet from 127.1.127.254:41413
I20260807 15:02:13.209867 8646 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: tablet copy: Beginning tablet copy session from remote peer at address 127.1.127.254:41413
I20260807 15:02:13.209875 8551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.209949 8551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.209998 8551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.210037 8551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.210557 8602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5eb970024e7249fe80916cba830ee843 (127.1.127.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.211488 8745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.211592 8745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:02:13.222062 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.211009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:52686)
0807 15:02:13.214802 (+ 3793us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.214827 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.214930 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.215118 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.215119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.215333 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.215403 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.215405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.216331 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.216332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.217197 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.217200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.217665 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.217666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.217736 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.218272 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.218278 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.218748 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.220754 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.220755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.220756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.221012 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.221283 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.221284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.221284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.221355 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.221653 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.221654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.221706 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.221817 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.221946 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3763,"thread_start_us":81,"threads_started":1}
I20260807 15:02:13.222041 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.211499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.1.127.254:41413)
0807 15:02:13.211541 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.211543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.214897 (+ 3354us) server_negotiation.cc:407] Connection header received
0807 15:02:13.215015 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.215016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.215030 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.215102 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.215482 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.216233 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.217286 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.217582 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.217655 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.218846 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.218847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.218848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.218857 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.220628 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.221141 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.221142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.221143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.221212 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.221439 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.221440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.221440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.221586 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.221654 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.221853 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.221946 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:02:13.222931 7986 tablet_copy_service.cc:140] P 649e709be6fc4b46b52150eb1b3f2e6c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a990961ebcc34988835efae4d61865b9 ({username='oryx', principal='oryx/127.1.127.252@KRBTEST.COM'} at 127.0.0.1:52686)
I20260807 15:02:13.223019 7986 tablet_copy_service.cc:161] P 649e709be6fc4b46b52150eb1b3f2e6c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a990961ebcc34988835efae4d61865b9 at {username='oryx', principal='oryx/127.1.127.252@KRBTEST.COM'} at 127.0.0.1:52686: session id = a990961ebcc34988835efae4d61865b9-00000000000000000000000000000000
I20260807 15:02:13.223526 7986 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:02:13.224069 8646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:02:13.225723 8646 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: tablet copy: Starting download of 0 data blocks...
I20260807 15:02:13.225875 8646 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: tablet copy: Starting download of 1 WAL segments...
I20260807 15:02:13.226828 8646 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:02:13.228210 7986 tablet_copy_service.cc:342] P 649e709be6fc4b46b52150eb1b3f2e6c: Request end of tablet copy session a990961ebcc34988835efae4d61865b9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.127.252@KRBTEST.COM'} at 127.0.0.1:52686
I20260807 15:02:13.228312 7986 tablet_copy_service.cc:433] P 649e709be6fc4b46b52150eb1b3f2e6c: ending tablet copy session a990961ebcc34988835efae4d61865b9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a990961ebcc34988835efae4d61865b9
I20260807 15:02:13.232115 8646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:13.233703 8755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.235899 8756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.236056 8758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.238889 8646 server_base.cc:1034] running on GCE node
I20260807 15:02:13.239027 8646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:13.239198 8646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:13.240363 8646 hybrid_clock.cc:648] HybridClock initialized: now 1786114933240329 us; error 48 us; skew 500 ppm
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:13.242182 8646 init.cc:377] Logged in from keytab as oryx/127.1.127.252@KRBTEST.COM (short username oryx)
I20260807 15:02:13.242547 8646 webserver.cc:460] Webserver started at http://127.1.127.252:46021/ using document root <none> and password file <none>
I20260807 15:02:13.242733 8646 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.242791 8646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.243673 8646 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:13.244345 8765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.244482 8646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:02:13.244525 8646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "a990961ebcc34988835efae4d61865b9"
format_stamp: "Formatted at 2026-08-07 15:02:13 on dist-test-slave-8tdl"
server_key: "181f68581da6554dc3fe205dbcfc890f"
server_key_iv: "fd858b05888d45f81a22f1bb330af5dd"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.244597 8646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.275003 8646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:13.277220 8646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:13.280864 8646 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.252:40673
I20260807 15:02:13.281267 8646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
I20260807 15:02:13.286868 8817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.252:40673 every 8 connection(s)
I20260807 15:02:13.287874 8818 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:02:13.288524 8818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Bootstrap starting.
I20260807 15:02:13.292164 8818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:02:13.292472 8818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: Bootstrap complete.
I20260807 15:02:13.292764 8818 raft_consensus.cc:348] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } }
I20260807 15:02:13.292873 8818 raft_consensus.cc:729] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a990961ebcc34988835efae4d61865b9, State: Initialized, Role: LEARNER
I20260807 15:02:13.292917 8818 consensus_queue.cc:260] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } }
I20260807 15:02:13.293159 8818 sys_catalog.cc:564] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:13.293191 8821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.293465 8821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:02:13.295003 8832 catalog_manager.cc:1259] Loaded cluster ID: de00166f4a8940c38f29adc42af1d309
I20260807 15:02:13.295249 8832 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: loading cluster ID for follower catalog manager: success
I20260807 15:02:13.296780 8832 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: acquiring CA information for follower catalog manager: success
I20260807 15:02:13.297153 8832 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:02:13.308925 8743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:13.309170 8743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:13.309198 8743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:13.310608 8743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:13.310670 8743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:13.310784 8743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:13.312700 8743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:35511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=40657
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:13.313040 8743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:13.313410 8743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:13.320601 8841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.320612 8839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.320773 8838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.321097 8743 server_base.cc:1034] running on GCE node
I20260807 15:02:13.321400 8743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:13.321676 8743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:13.322824 8743 hybrid_clock.cc:648] HybridClock initialized: now 1786114933322814 us; error 35 us; skew 500 ppm
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:13.325785 8743 init.cc:377] Logged in from keytab as oryx/127.1.127.193@KRBTEST.COM (short username oryx)
I20260807 15:02:13.326287 8743 webserver.cc:460] Webserver started at http://127.1.127.193:40657/ using document root <none> and password file <none>
I20260807 15:02:13.326537 8743 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.326601 8743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.328088 8743 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:13.328904 8848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.329214 8743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:13.329355 8743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "6a766c4220014ba796fa0a75c7b08f53"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "d7a1004054466c7e0a75c6c34a859341"
server_key_iv: "3d71de9fe9351493d8bcee063d711167"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.329929 8743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.347071 8743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:13.349797 8743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:13.349946 8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:13.350154 8743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:13.350575 8743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:13.350617 8743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.350643 8743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:13.350657 8743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.357523 8743 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35511
I20260807 15:02:13.357602 8968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35511 every 8 connection(s)
I20260807 15:02:13.358115 8743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:13.365408 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8743
I20260807 15:02:13.365578 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8170
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:13.370958 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:53509 (local address 127.1.127.254:41413)
0807 15:02:13.359164 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.359167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.359404 (+ 237us) server_negotiation.cc:407] Connection header received
0807 15:02:13.360096 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.360098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.360114 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.360177 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.361072 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.361832 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363199 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363480 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363554 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.367631 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.367633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.367633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.367641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.369280 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.369848 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.369849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.369850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.369917 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.370241 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.370242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.370242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.370375 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.370410 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.370747 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.370840 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:02:13.371254 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:53509)
0807 15:02:13.359253 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.359299 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.359988 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.360677 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.360678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.360827 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.360959 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.360960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.361911 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.361911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.363112 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.363115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363555 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.363621 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.364188 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.364196 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.367509 (+ 3313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.369385 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.369394 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.369397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.369741 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.369993 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.369994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.369994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.370153 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.370450 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.370455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.370585 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.370737 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.370987 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":82,"thread_start_us":174,"threads_started":1}
I20260807 15:02:13.371747 8970 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:13.371877 8970 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.372123 8970 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:13.372632 7966 ts_manager.cc:194] Re-registered known tserver with Master: 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511)
I20260807 15:02:13.373127 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.1.127.193:49339)
0807 15:02:13.359545 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.359557 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.359993 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.360420 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.360422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.360772 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.360954 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.360960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.362135 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.362136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.362921 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.362924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363387 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.363532 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.364265 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.364271 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.368386 (+ 4115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.370809 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.370810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.370811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.371647 (+ 836us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.371921 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.371922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.371923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.371992 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.372489 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.372490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.372547 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.372622 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.372771 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260807 15:02:13.373395 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:53509
I20260807 15:02:13.373944 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:49339 (local address 127.1.127.252:40673)
0807 15:02:13.359752 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.359754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.359761 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:13.360283 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.360284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.360298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.360391 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.361072 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.362061 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363027 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363301 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363366 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.368514 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.368516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.368516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.368526 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.370623 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.371751 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.371752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.371753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.371845 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.372070 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.372071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.372071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.372273 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.373661 (+ 1388us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.373717 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.373840 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260807 15:02:13.374377 8969 heartbeater.cc:347] Connected to a master server at 127.1.127.252:40673
I20260807 15:02:13.374437 8969 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.374511 8969 heartbeater.cc:511] Master 127.1.127.252:40673 requested a full tablet report, sending...
I20260807 15:02:13.374569 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:37837
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:41413
--webserver_port=45281
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:02:13.374997 8782 ts_manager.cc:194] Registered new tserver with Master: 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511)
I20260807 15:02:13.378559 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:36563 (local address 127.1.127.253:38271)
0807 15:02:13.359189 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.359192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.359382 (+ 190us) server_negotiation.cc:407] Connection header received
0807 15:02:13.360173 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.360174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.360187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.360243 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.361065 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.361619 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363244 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363579 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363655 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.375305 (+ 11650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.375376 (+ 71us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.375377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.375386 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.377070 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.377584 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.377585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.377585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.377646 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.377892 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.377894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.377894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.378161 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.378230 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.378411 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.378492 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:02:13.378857 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.359000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.1.127.193:36563)
0807 15:02:13.359255 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.359299 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.359974 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.360281 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.360285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.360697 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.360954 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.360959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.361740 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.361741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.363122 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.363124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.363665 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.363666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.363747 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.364491 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.364496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.375159 (+ 10663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.377190 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.377191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.377192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.377501 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.377720 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.377721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.377721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.377800 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.378239 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.378240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.378299 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.378392 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.378623 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":65,"thread_start_us":152,"threads_started":1}
I20260807 15:02:13.379410 8971 heartbeater.cc:347] Connected to a master server at 127.1.127.253:38271
I20260807 15:02:13.379491 8971 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.379565 8971 heartbeater.cc:511] Master 127.1.127.253:38271 requested a full tablet report, sending...
I20260807 15:02:13.379958 8592 ts_manager.cc:194] Registered new tserver with Master: 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511)
W20260807 15:02:13.475400 8977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:13.475684 8977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:13.475715 8977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:13.477216 8977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:13.477309 8977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:13.477365 8977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:13.479435 8977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:37837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=45281
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:13.479773 8977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:13.480046 8977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:13.486060 8982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.486212 8986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.486225 8983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.486608 8977 server_base.cc:1034] running on GCE node
I20260807 15:02:13.486902 8977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:13.487210 8977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:13.488397 8977 hybrid_clock.cc:648] HybridClock initialized: now 1786114933488349 us; error 78 us; skew 500 ppm
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:13.491528 8977 init.cc:377] Logged in from keytab as oryx/127.1.127.194@KRBTEST.COM (short username oryx)
I20260807 15:02:13.492102 8977 webserver.cc:460] Webserver started at http://127.1.127.194:45281/ using document root <none> and password file <none>
I20260807 15:02:13.492326 8977 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.492386 8977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.494028 8977 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:13.494796 8993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.495100 8977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:02:13.495217 8977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "ca89db920c514d6794bfdec0c0b56014"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "1f5533d0d72de47e6de90219c9a62767"
server_key_iv: "88d67439a7aef60ba1ce7fdeac0a372f"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.495604 8977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.530405 8977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:13.533329 8977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:13.533555 8977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:13.533814 8977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:13.534196 8977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:13.534235 8977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.534294 8977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:13.534315 8977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.541877 8977 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:37837
I20260807 15:02:13.542026 9113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:37837 every 8 connection(s)
I20260807 15:02:13.542407 8977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:13.552471 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 8977
I20260807 15:02:13.552664 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8310
I20260807 15:02:13.555351 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:33027 (local address 127.1.127.252:40673)
0807 15:02:13.543550 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.543552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.544024 (+ 472us) server_negotiation.cc:407] Connection header received
0807 15:02:13.545187 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.545189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.545206 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.545255 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.546160 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.546953 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548106 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.548434 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548470 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.551121 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.551122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.551122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.551130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.553170 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554044 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554451 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554604 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.554651 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.555148 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.555245 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:02:13.555351 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:40361 (local address 127.1.127.254:41413)
0807 15:02:13.543489 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.543491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.543871 (+ 380us) server_negotiation.cc:407] Connection header received
0807 15:02:13.545120 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.545122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.545136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.545207 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.546159 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.546962 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548115 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.548416 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548460 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.551572 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.551573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.551573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.551582 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.553551 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554045 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554459 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554604 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.554651 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.555148 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.555244 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:02:13.555392 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:42723 (local address 127.1.127.253:38271)
0807 15:02:13.543981 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.543983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.544045 (+ 62us) server_negotiation.cc:407] Connection header received
0807 15:02:13.545119 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.545121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.545136 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.545207 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.546140 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.546659 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.547793 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.547991 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548075 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.550739 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.550741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.550741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.550751 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.552212 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554051 (+ 1839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554124 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554450 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.554451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.554451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.554637 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.554722 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.555139 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.555250 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:02:13.555976 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:40361)
0807 15:02:13.543704 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.543712 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.544965 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.545290 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.545296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.545830 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.546041 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.546047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.547062 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.547062 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.548000 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.548002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.548546 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.548622 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.549310 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.549314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.551470 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.553641 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.553642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.553643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.553950 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554190 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.554191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.554365 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554737 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.554746 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.554976 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.555460 (+ 484us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.555753 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":186,"thread_start_us":102,"threads_started":1}
I20260807 15:02:13.555976 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.1.127.194:33027)
0807 15:02:13.543620 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.543665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.545125 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.545301 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.545302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.545830 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.546041 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.546047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.547059 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.547060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.548000 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.548002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.548548 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.548622 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.549086 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.549090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.550975 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.553269 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.553270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.553271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.553959 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554190 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.554191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.554365 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554737 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.554746 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.554976 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.555460 (+ 484us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.555703 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":135,"thread_start_us":103,"threads_started":1}
I20260807 15:02:13.556731 9115 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:13.556763 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.543764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.1.127.194:42723)
0807 15:02:13.543938 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.543949 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.544971 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.545296 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.545297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.545804 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.546040 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.546046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.546801 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.546801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.547680 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.547682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.548082 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.548084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.548217 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.548768 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.548777 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.550549 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.552277 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.552285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.552288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.553638 (+ 1350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554209 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.554211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.554211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.554371 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.554774 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.554775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.554976 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.556448 (+ 1472us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.556594 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260807 15:02:13.556869 9115 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.557082 9115 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:13.557556 7966 ts_manager.cc:194] Re-registered known tserver with Master: ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:13.558111 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:40361
I20260807 15:02:13.559281 9116 heartbeater.cc:347] Connected to a master server at 127.1.127.253:38271
I20260807 15:02:13.559422 9116 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.559520 9116 heartbeater.cc:511] Master 127.1.127.253:38271 requested a full tablet report, sending...
I20260807 15:02:13.559719 9114 heartbeater.cc:347] Connected to a master server at 127.1.127.252:40673
I20260807 15:02:13.559778 9114 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.559835 9114 heartbeater.cc:511] Master 127.1.127.252:40673 requested a full tablet report, sending...
I20260807 15:02:13.559906 8592 ts_manager.cc:194] Registered new tserver with Master: ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:13.560130 8782 ts_manager.cc:194] Registered new tserver with Master: ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:13.560981 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:38737
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:41413
--webserver_port=41699
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:13.649886 9121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:13.650127 9121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:13.650156 9121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:13.651453 9121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:13.651503 9121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:13.651532 9121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:13.653070 9121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:38737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=41699
--tserver_master_addrs=127.1.127.254:41413,127.1.127.253:38271,127.1.127.252:40673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:13.653288 9121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:13.653491 9121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:13.658962 9126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.659303 9121 server_base.cc:1034] running on GCE node
W20260807 15:02:13.658969 9127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:13.659011 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:13.659677 9121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:13.659948 9121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:13.661149 9121 hybrid_clock.cc:648] HybridClock initialized: now 1786114933661121 us; error 39 us; skew 500 ppm
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:13.664263 9121 init.cc:377] Logged in from keytab as oryx/127.1.127.195@KRBTEST.COM (short username oryx)
I20260807 15:02:13.664901 9121 webserver.cc:460] Webserver started at http://127.1.127.195:41699/ using document root <none> and password file <none>
I20260807 15:02:13.665162 9121 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:13.665256 9121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:13.667016 9121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:13.667707 9137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.667898 9121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:13.667989 9121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6cf09af3607c4e79a64d456d6d496c97"
format_stamp: "Formatted at 2026-08-07 15:02:12 on dist-test-slave-8tdl"
server_key: "f031089d681b7827f0e42a5ebbd93ab2"
server_key_iv: "ffaa7df0f51abf69f9502234ea800b3c"
server_key_version: "encryptionkey@0"
I20260807 15:02:13.668404 9121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:13.677155 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.674257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:60122)
0807 15:02:13.674377 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.674392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.674460 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.674651 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.674652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.674654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:13.674733 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.674736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.675444 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.675446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.676175 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.676177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.676852 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.676853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.676933 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.676938 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.677017 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.677068 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:02:13.677176 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.674326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.1.127.252:40673)
0807 15:02:13.674398 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.674401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.674439 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:02:13.674539 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.674540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.674569 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.674644 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:13.674798 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.675348 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.676269 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.676748 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.676849 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.676911 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.677047 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.677089 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:02:13.677558 8786 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 15:02:13.677826 8821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 649e709be6fc4b46b52150eb1b3f2e6c. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.677937 8821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:02:13.685549 9121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:13.688159 9121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:13.688342 9121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:13.688603 9121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:13.688930 9121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:13.688980 9121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.689016 9121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:13.689066 9121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:13.696190 9121 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38737
I20260807 15:02:13.696278 9254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38737 every 8 connection(s)
I20260807 15:02:13.696698 9121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:13.697158 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 9121
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.252@KRBTEST.COM
Aug 07 15:02:13 dist-test-slave-8tdl krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114933, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.253@KRBTEST.COM
I20260807 15:02:13.712601 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.697548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:57137)
0807 15:02:13.698028 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.698087 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.699742 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.700107 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.700111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.700649 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.701036 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.701042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.702131 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.702132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.703677 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.703680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704239 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.704385 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.705042 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.705054 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.707596 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.710484 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.710496 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.710499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.710886 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.711112 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.711113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.711113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.711272 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.711646 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.711655 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.711845 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.711952 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.712297 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":132,"thread_start_us":92,"threads_started":1}
I20260807 15:02:13.713253 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.697638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:57137 (local address 127.1.127.254:41413)
0807 15:02:13.697704 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.697706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.698213 (+ 507us) server_negotiation.cc:407] Connection header received
0807 15:02:13.699932 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.699934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.699949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.700005 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.701131 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.701982 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.703799 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704114 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704177 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.707717 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.707718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.707719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.707727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.710361 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.710972 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.710973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.710974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.711040 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.711347 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.711347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.711348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.711558 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.711622 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.712161 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.712258 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:02:13.713702 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.698064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.1.127.195:49105)
0807 15:02:13.698584 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.698596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.699887 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.700132 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.700133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.701040 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.701124 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.701126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.702121 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.702122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.703671 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.703674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704228 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.704385 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.705759 (+ 1374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.705764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.709072 (+ 3308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.711645 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.711646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.711647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.712177 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.712420 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.712420 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.712421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.712901 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.713274 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.713275 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.713323 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.713386 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.713528 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":159,"thread_start_us":92,"threads_started":1}
I20260807 15:02:13.714203 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.698308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:49105 (local address 127.1.127.253:38271)
0807 15:02:13.698373 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.698376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.698706 (+ 330us) server_negotiation.cc:407] Connection header received
0807 15:02:13.700012 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.700012 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.700025 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.700055 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.701221 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.701959 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.703781 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704104 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704163 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.709157 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.709158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.709159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.709167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.711555 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.712256 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.712257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.712257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.712322 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.713012 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.713013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.713014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.713180 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.713244 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.714001 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.714098 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:02:13.714529 9256 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41413
I20260807 15:02:13.714617 9257 heartbeater.cc:347] Connected to a master server at 127.1.127.253:38271
I20260807 15:02:13.714643 9256 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.714716 9257 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.714869 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.697608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:40915 (local address 127.1.127.252:40673)
0807 15:02:13.697663 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.697665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.698331 (+ 666us) server_negotiation.cc:407] Connection header received
0807 15:02:13.699922 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.699923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.699942 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.699974 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.701152 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.701977 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.703797 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704105 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704151 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.708207 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.708209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.708209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.708218 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.710126 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.713709 (+ 3583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.713711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.713711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.713786 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.714034 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.714036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.714036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.714184 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.714244 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.714654 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.714770 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:02:13.714938 9256 heartbeater.cc:511] Master 127.1.127.254:41413 requested a full tablet report, sending...
I20260807 15:02:13.714941 9257 heartbeater.cc:511] Master 127.1.127.253:38271 requested a full tablet report, sending...
I20260807 15:02:13.715167 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.697548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.1.127.195:40915)
0807 15:02:13.698248 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.698260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.699742 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.700107 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.700111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.700649 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:13.701036 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.701043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.702121 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.702122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.703671 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.703674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704234 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.704385 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.705651 (+ 1266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:13.705657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:13.708092 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:13.713168 (+ 5076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.713170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.713170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.713423 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.713865 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:13.713866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:13.713866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:13.713933 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:13.714278 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:13.714279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:13.714460 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.714944 (+ 484us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.715056 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":335,"thread_start_us":94,"threads_started":1}
I20260807 15:02:13.715343 7966 ts_manager.cc:194] Re-registered known tserver with Master: 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737)
I20260807 15:02:13.715438 8592 ts_manager.cc:194] Registered new tserver with Master: 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737)
I20260807 15:02:13.715919 7966 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:57137
I20260807 15:02:13.716638 9255 heartbeater.cc:347] Connected to a master server at 127.1.127.252:40673
I20260807 15:02:13.716701 9255 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:13.716784 9255 heartbeater.cc:511] Master 127.1.127.252:40673 requested a full tablet report, sending...
I20260807 15:02:13.717777 8782 ts_manager.cc:194] Registered new tserver with Master: 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737)
I20260807 15:02:13.717880 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.698432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.1.127.254:41413)
0807 15:02:13.701328 (+ 2896us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.701330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.701337 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:13.701358 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.701359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.701371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.702153 (+ 782us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.702191 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.703000 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704213 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.704497 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.704852 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.707260 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.707262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.707262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.707271 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.710869 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.711300 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.711302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.711302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.711359 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.711588 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.711589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.711589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.716593 (+ 5004us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.717543 (+ 950us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.717591 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.717710 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2864,"thread_start_us":91,"threads_started":1}
I20260807 15:02:13.721091 7966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:13.723932 7966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:13.726094 8748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: NON_VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: true } } }
I20260807 15:02:13.726215 8748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.727098 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.699205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.1.127.253:38271)
0807 15:02:13.699600 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.699603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.699613 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:13.699646 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.699647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.699663 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.701239 (+ 1576us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.704996 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.707133 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.708223 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.708534 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.708604 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.717978 (+ 9374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.717980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.717980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.717990 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.724996 (+ 7006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.725799 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.725801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.725802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.725911 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.726166 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.726167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.726168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.726328 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.726844 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.726891 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.726989 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":95,"threads_started":1}
I20260807 15:02:13.727978 9274 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: attempting to promote NON_VOTER a990961ebcc34988835efae4d61865b9 to VOTER
I20260807 15:02:13.728106 9274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } }
I20260807 15:02:13.728813 8784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 LEARNER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:02:13.728905 8602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Refusing update from remote peer 649e709be6fc4b46b52150eb1b3f2e6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:02:13.729048 8748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 15:02:13.729287 8748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 15:02:13.729943 9275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a990961ebcc34988835efae4d61865b9 (127.1.127.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.730089 8602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a990961ebcc34988835efae4d61865b9 (127.1.127.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.731074 9275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.731166 9275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.731180 9274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.731248 9274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.731920 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.699406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.1.127.252:40673)
0807 15:02:13.700227 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.700229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.700236 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:13.700256 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.700264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.700277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.700439 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:13.705491 (+ 5052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.706319 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.709285 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.709619 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.709688 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.725784 (+ 16096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:13.725786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:13.725786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:13.725796 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:13.730210 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.730667 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.730669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.730669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.730811 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:13.731027 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:13.731028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:13.731028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:13.731173 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:13.731272 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.731717 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.731807 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":190,"threads_started":1}
I20260807 15:02:13.733850 8748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.733969 8748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:13.734517 8784 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a990961ebcc34988835efae4d61865b9 (127.1.127.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.735971 8821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.736083 8821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:02:13.736563 9283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:13.736691 9283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:02:13.738376 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.732853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37837 (local address 127.0.0.1:52598)
0807 15:02:13.732911 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.732935 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.732976 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.733588 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.733589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.733591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:13.733648 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.733650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.735222 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.735222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.736529 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.736534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.737920 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.737922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.738077 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.738083 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.738211 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.738284 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":37}
I20260807 15:02:13.738705 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.732973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38737 (local address 127.0.0.1:54484)
0807 15:02:13.733138 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.733149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.733281 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.733832 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.733833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.733836 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:13.733889 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.733891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.735309 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.735310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.736579 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.736583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.738252 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.738253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.738323 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.738327 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.738357 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.738403 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260807 15:02:13.739447 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.733347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.1.127.195:38737)
0807 15:02:13.733527 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.733532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.733544 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:13.733586 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.733588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.733606 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.733750 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:13.734079 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.735204 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.736928 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.738125 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.739077 (+ 952us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.739175 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.739205 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.739311 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260807 15:02:13.739504 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.732854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35511 (local address 127.0.0.1:58450)
0807 15:02:13.734212 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:13.734223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:13.734489 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:13.734891 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:13.734892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:13.734894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:13.734942 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.734943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.736352 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.736353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.737790 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:13.737795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.739221 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.739222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:13.739292 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.739298 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:02:13.739365 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:02:13.739416 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":1196,"thread_start_us":109,"threads_started":1}
I20260807 15:02:13.739589 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.733579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.1.127.193:35511)
0807 15:02:13.733761 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.733766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.734554 (+ 788us) server_negotiation.cc:407] Connection header received
0807 15:02:13.734606 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.734608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.734624 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.734803 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:13.735013 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.736244 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.737904 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.738911 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.739002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.739111 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.739427 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.739484 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260807 15:02:13.739686 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:13.733010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.1.127.194:37837)
0807 15:02:13.733203 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:13.733209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:13.733220 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:13.733268 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:13.733270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:13.733291 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:13.733788 (+ 497us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:13.733864 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.735128 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.736689 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:13.737801 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:13.738627 (+ 826us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:13.739417 (+ 790us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:13.739447 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 15:02:13.739500 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260807 15:02:13.740501 9045 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b (DEFAULT_TABLE table=test-table [id=fbd98cfa54fb4d4a91ef3cfdabc8b497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:13.740527 8898 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b (DEFAULT_TABLE table=test-table [id=fbd98cfa54fb4d4a91ef3cfdabc8b497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:13.740913 8898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f9391fb8745b5b95a070d4c88ee1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:13.740921 9045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f9391fb8745b5b95a070d4c88ee1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:13.740871 9186 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b (DEFAULT_TABLE table=test-table [id=fbd98cfa54fb4d4a91ef3cfdabc8b497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:13.741278 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3f9391fb8745b5b95a070d4c88ee1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:13.745154 9290 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Bootstrap starting.
I20260807 15:02:13.745728 9292 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Bootstrap starting.
I20260807 15:02:13.745883 9291 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Bootstrap starting.
I20260807 15:02:13.746274 9292 tablet_bootstrap.cc:654] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:13.746234 9290 tablet_bootstrap.cc:654] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:13.746546 9292 log.cc:824] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:13.746605 9291 tablet_bootstrap.cc:654] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:13.746872 9290 log.cc:824] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:13.746944 9291 log.cc:824] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:13.747778 9290 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: No bootstrap required, opened a new log
I20260807 15:02:13.747861 9290 ts_tablet_manager.cc:1397] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:02:13.747922 9292 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: No bootstrap required, opened a new log
I20260807 15:02:13.747962 9291 tablet_bootstrap.cc:492] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: No bootstrap required, opened a new log
I20260807 15:02:13.747993 9292 ts_tablet_manager.cc:1397] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:02:13.748023 9291 ts_tablet_manager.cc:1397] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:13.749620 9292 raft_consensus.cc:348] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.749788 9292 raft_consensus.cc:374] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:13.749832 9292 raft_consensus.cc:729] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a766c4220014ba796fa0a75c7b08f53, State: Initialized, Role: FOLLOWER
I20260807 15:02:13.749951 9292 consensus_queue.cc:260] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.750237 9292 ts_tablet_manager.cc:1428] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:13.750316 9291 raft_consensus.cc:348] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.750483 9291 raft_consensus.cc:374] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:13.750528 9291 raft_consensus.cc:729] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf09af3607c4e79a64d456d6d496c97, State: Initialized, Role: FOLLOWER
I20260807 15:02:13.750540 8970 heartbeater.cc:503] Master 127.1.127.254:41413 was elected leader, sending a full tablet report...
I20260807 15:02:13.750658 9291 consensus_queue.cc:260] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.751084 9291 ts_tablet_manager.cc:1428] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 15:02:13.751372 9256 heartbeater.cc:503] Master 127.1.127.254:41413 was elected leader, sending a full tablet report...
I20260807 15:02:13.751551 9290 raft_consensus.cc:348] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.751710 9290 raft_consensus.cc:374] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:13.751745 9290 raft_consensus.cc:729] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca89db920c514d6794bfdec0c0b56014, State: Initialized, Role: FOLLOWER
I20260807 15:02:13.751871 9290 consensus_queue.cc:260] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:13.752169 9290 ts_tablet_manager.cc:1428] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:02:13.752287 9115 heartbeater.cc:503] Master 127.1.127.254:41413 was elected leader, sending a full tablet report...
W20260807 15:02:13.793531 9117 tablet.cc:2369] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:02:13.859333 8972 tablet.cc:2369] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:02:13.948832 9258 tablet.cc:2369] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:14.108729 9296 raft_consensus.cc:482] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:14.108922 9296 raft_consensus.cc:504] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:14.109344 9296 leader_election.cc:288] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737), ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:14.113986 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.109720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:36661 (local address 127.1.127.195:38737)
0807 15:02:14.109777 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.109780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.110006 (+ 226us) server_negotiation.cc:407] Connection header received
0807 15:02:14.110093 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.110095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.110107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.110179 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:14.110453 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.111241 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.112943 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.113555 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.113620 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.113626 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.113856 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.113911 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:02:14.113987 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.109720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:50821 (local address 127.1.127.194:37837)
0807 15:02:14.109804 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.109806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.109924 (+ 118us) server_negotiation.cc:407] Connection header received
0807 15:02:14.110113 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.110114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.110124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.110179 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:14.110453 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.111286 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.112979 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.113522 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.113616 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.113626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.113856 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.113909 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:02:14.113977 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.109624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38737 (local address 127.1.127.193:36661)
0807 15:02:14.109832 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:14.109842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:14.109997 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:14.110206 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:14.110207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:14.110210 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:14.110336 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:14.110339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.111388 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.111389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:14.112779 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:14.112783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.113642 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.113644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:14.113734 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.113742 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:02:14.113806 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:02:14.113862 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":98,"thread_start_us":68,"threads_started":1}
I20260807 15:02:14.113976 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.109624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37837 (local address 127.1.127.193:50821)
0807 15:02:14.109818 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:14.109832 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:14.110025 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:14.110206 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:14.110207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:14.110211 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:14.110336 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:14.110340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.111388 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.111389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:14.112780 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:14.112783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.113645 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.113645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:14.113734 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.113741 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:02:14.113806 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:02:14.113862 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":113,"thread_start_us":71,"threads_started":1}
I20260807 15:02:14.114444 9208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f9391fb8745b5b95a070d4c88ee1b" candidate_uuid: "6a766c4220014ba796fa0a75c7b08f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf09af3607c4e79a64d456d6d496c97" is_pre_election: true
I20260807 15:02:14.114645 9208 raft_consensus.cc:2454] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a766c4220014ba796fa0a75c7b08f53 in term 0.
I20260807 15:02:14.114612 9065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f9391fb8745b5b95a070d4c88ee1b" candidate_uuid: "6a766c4220014ba796fa0a75c7b08f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca89db920c514d6794bfdec0c0b56014" is_pre_election: true
I20260807 15:02:14.114773 9065 raft_consensus.cc:2454] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a766c4220014ba796fa0a75c7b08f53 in term 0.
I20260807 15:02:14.115051 8851 leader_election.cc:302] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a766c4220014ba796fa0a75c7b08f53, 6cf09af3607c4e79a64d456d6d496c97; no voters:
I20260807 15:02:14.115196 9296 raft_consensus.cc:2781] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:14.115257 9296 raft_consensus.cc:482] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:14.115280 9296 raft_consensus.cc:3037] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:14.116091 9296 raft_consensus.cc:504] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:14.116248 9296 leader_election.cc:288] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [CANDIDATE]: Term 1 election: Requested vote from peers 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737), ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:14.116488 9065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f9391fb8745b5b95a070d4c88ee1b" candidate_uuid: "6a766c4220014ba796fa0a75c7b08f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca89db920c514d6794bfdec0c0b56014"
I20260807 15:02:14.116472 9208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3f9391fb8745b5b95a070d4c88ee1b" candidate_uuid: "6a766c4220014ba796fa0a75c7b08f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cf09af3607c4e79a64d456d6d496c97"
I20260807 15:02:14.116583 9065 raft_consensus.cc:3037] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:14.116567 9208 raft_consensus.cc:3037] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:14.117331 9065 raft_consensus.cc:2454] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a766c4220014ba796fa0a75c7b08f53 in term 1.
I20260807 15:02:14.117489 9208 raft_consensus.cc:2454] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a766c4220014ba796fa0a75c7b08f53 in term 1.
I20260807 15:02:14.117581 8849 leader_election.cc:302] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a766c4220014ba796fa0a75c7b08f53, ca89db920c514d6794bfdec0c0b56014; no voters:
I20260807 15:02:14.117684 9296 raft_consensus.cc:2781] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:14.117799 9296 raft_consensus.cc:686] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 1 LEADER]: Becoming Leader. State: Replica: 6a766c4220014ba796fa0a75c7b08f53, State: Running, Role: LEADER
I20260807 15:02:14.117895 9296 consensus_queue.cc:237] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:14.118808 7964 catalog_manager.cc:5495] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "6a766c4220014ba796fa0a75c7b08f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:14.186494 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.184040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.1.127.193:35511)
0807 15:02:14.184100 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.184102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.184125 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:02:14.184162 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.184163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.184182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.184237 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:14.184404 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.184913 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.185713 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.185936 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.186007 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.186198 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:14.186307 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:14.186373 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.186407 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.186431 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:02:14.188534 9208 raft_consensus.cc:1261] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Refusing update from remote peer 6a766c4220014ba796fa0a75c7b08f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:14.188545 9065 raft_consensus.cc:1261] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Refusing update from remote peer 6a766c4220014ba796fa0a75c7b08f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:14.188922 9301 consensus_queue.cc:1035] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:14.189117 9296 consensus_queue.cc:1035] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:14.193516 9305 mvcc.cc:205] Tried to move back new op lower bound from 7315926770433335296 to 7315926770147479552. Current Snapshot: MvccSnapshot[applied={T|T < 7315926770433335296}]
I20260807 15:02:14.194850 9306 mvcc.cc:205] Tried to move back new op lower bound from 7315926770433335296 to 7315926770147479552. Current Snapshot: MvccSnapshot[applied={T|T < 7315926770433335296}]
I20260807 15:02:14.195381 7964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:14.195475 7964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:14.197074 7964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 649e709be6fc4b46b52150eb1b3f2e6c: Sending DeleteTablet for 3 replicas of tablet 2b3f9391fb8745b5b95a070d4c88ee1b
I20260807 15:02:14.197451 9045 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:14 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:52598
I20260807 15:02:14.197588 8898 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:14 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:58450
I20260807 15:02:14.197724 9186 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3f9391fb8745b5b95a070d4c88ee1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:14 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:54484
I20260807 15:02:14.198004 9313 tablet_replica.cc:330] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: stopping tablet replica
I20260807 15:02:14.198122 9313 raft_consensus.cc:2229] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:14.198258 9313 raft_consensus.cc:2258] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:14.199340 9313 ts_tablet_manager.cc:1905] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:14.199608 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 7933
I20260807 15:02:14.203120 9313 ts_tablet_manager.cc:1918] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:14.203213 9313 log.cc:1196] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/2b3f9391fb8745b5b95a070d4c88ee1b
I20260807 15:02:14.203509 9313 ts_tablet_manager.cc:1939] T 2b3f9391fb8745b5b95a070d4c88ee1b P ca89db920c514d6794bfdec0c0b56014: Deleting consensus metadata
I20260807 15:02:14.203745 9316 tablet_replica.cc:330] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: stopping tablet replica
I20260807 15:02:14.203881 9316 raft_consensus.cc:2229] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:14.204005 9316 raft_consensus.cc:2258] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:14.204926 9316 ts_tablet_manager.cc:1905] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:14.206951 9316 ts_tablet_manager.cc:1918] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:14.207036 9316 log.cc:1196] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/2b3f9391fb8745b5b95a070d4c88ee1b
I20260807 15:02:14.207324 9316 ts_tablet_manager.cc:1939] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6cf09af3607c4e79a64d456d6d496c97: Deleting consensus metadata
W20260807 15:02:14.208575 9141 connection.cc:535] server connection from 127.0.0.1:54484 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:02:14.208614 8996 connection.cc:535] server connection from 127.0.0.1:52598 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:02:14.209146 9314 tablet_replica.cc:330] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: stopping tablet replica
I20260807 15:02:14.209285 9314 raft_consensus.cc:2229] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:14.209359 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.198791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.1.127.252:40673)
0807 15:02:14.198835 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.198838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.198848 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:14.198905 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.198906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.198920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.198976 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:14.200621 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.201524 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.202823 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.203186 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.203239 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.205608 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:14.205610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:14.205611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:14.205619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:14.207631 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.208206 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.208208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.208208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.208275 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.208529 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.208530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.208530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.208686 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:14.208771 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.209150 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.209257 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:02:14.209407 9314 raft_consensus.cc:2258] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:14.211123 9314 ts_tablet_manager.cc:1905] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:14.212605 9314 ts_tablet_manager.cc:1918] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:14.212695 9314 log.cc:1196] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/2b3f9391fb8745b5b95a070d4c88ee1b
I20260807 15:02:14.213078 9314 ts_tablet_manager.cc:1939] T 2b3f9391fb8745b5b95a070d4c88ee1b P 6a766c4220014ba796fa0a75c7b08f53: Deleting consensus metadata
W20260807 15:02:14.213528 8849 connection.cc:441] server connection from 127.0.0.1:58450 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58450 (request call id 1) could send its response
I20260807 15:02:14.220058 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.212460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.1.127.252:40673)
0807 15:02:14.212501 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.212502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.212513 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:14.212542 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.212543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.212554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.212617 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:14.213702 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.214207 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.215259 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.215497 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.215560 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.217047 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:14.217049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:14.217049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:14.217056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:14.218937 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.219284 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.219285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.219286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.219347 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.219528 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.219529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.219529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.219646 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:14.219742 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.219851 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.219934 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:02:14.221299 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:14.212422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46332 (local address 127.1.127.253:38271)
0807 15:02:14.212505 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:14.212507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:14.212518 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:14.212546 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:14.212548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:14.212565 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:14.212633 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:14.213782 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.214662 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.215862 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:14.216229 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:14.216274 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:14.217839 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:14.217840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:14.217841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:14.217849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:14.219861 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.220302 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.220303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.220303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.220370 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:14.220698 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:14.220699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:14.220700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:14.220830 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:14.220914 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:14.221111 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:02:14.221189 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:02:15.194366 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.193975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:40187)
0807 15:02:15.194207 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.194304 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":75,"threads_started":1}
I20260807 15:02:15.194594 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.194327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:60471)
0807 15:02:15.194471 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.194540 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260807 15:02:15.194617 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.194045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:46155)
0807 15:02:15.194399 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.194486 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":78,"threads_started":1}
W20260807 15:02:15.194864 8970 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
W20260807 15:02:15.194896 9115 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
W20260807 15:02:15.194996 9256 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
I20260807 15:02:15.697770 9327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 649e709be6fc4b46b52150eb1b3f2e6c)
I20260807 15:02:15.697919 9327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } }
I20260807 15:02:15.698127 9328 raft_consensus.cc:482] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 649e709be6fc4b46b52150eb1b3f2e6c)
I20260807 15:02:15.698273 9328 raft_consensus.cc:504] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } }
I20260807 15:02:15.698550 9327 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413), a990961ebcc34988835efae4d61865b9 (127.1.127.252:40673)
I20260807 15:02:15.698588 9328 leader_election.cc:288] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413), 5eb970024e7249fe80916cba830ee843 (127.1.127.253:38271)
I20260807 15:02:15.699096 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34242)
0807 15:02:15.698997 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.699064 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":278,"threads_started":1}
I20260807 15:02:15.699187 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34248)
0807 15:02:15.699070 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.699137 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":73,"threads_started":1}
W20260807 15:02:15.699282 8574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
W20260807 15:02:15.699311 8766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
I20260807 15:02:15.699841 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.699754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34258)
0807 15:02:15.699790 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.699815 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 15:02:15.699865 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.699777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34264)
0807 15:02:15.699814 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.699838 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:02:15.699947 8766 leader_election.cc:334] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413): Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
W20260807 15:02:15.700054 8574 leader_election.cc:334] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413): Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
I20260807 15:02:15.702301 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:40673 (local address 127.0.0.1:41900)
0807 15:02:15.698694 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.698708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.698766 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.699223 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.699225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.699227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.699300 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.699304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.700402 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.700403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.701315 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.701318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.701994 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.701996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.702080 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.702084 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.702154 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.702203 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":44,"threads_started":1}
I20260807 15:02:15.702355 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:38271 (local address 127.0.0.1:35002)
0807 15:02:15.698861 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.698872 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.698920 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.699647 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.699649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.699651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.699713 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.699716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.700595 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.700596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.701466 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.701469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.702011 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.702012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.702079 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.702083 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.702173 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.702221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260807 15:02:15.702425 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.1.127.252:40673)
0807 15:02:15.699025 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.699027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.699038 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:15.699074 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.699075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.699088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.699382 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.699425 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.700296 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.701467 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.701892 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.701979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.701993 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.702160 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.702216 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260807 15:02:15.702641 8784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5eb970024e7249fe80916cba830ee843" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a990961ebcc34988835efae4d61865b9" is_pre_election: true
I20260807 15:02:15.702801 8784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eb970024e7249fe80916cba830ee843 in term 1.
I20260807 15:02:15.702920 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.698872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.1.127.253:38271)
0807 15:02:15.699342 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.699344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.699354 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:15.699414 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.699415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.699427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.699779 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.699816 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.700505 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.701555 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.701934 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.702010 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.702022 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.702181 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.702545 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":95,"threads_started":1}
I20260807 15:02:15.702988 8575 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5eb970024e7249fe80916cba830ee843, a990961ebcc34988835efae4d61865b9; no voters: 649e709be6fc4b46b52150eb1b3f2e6c
I20260807 15:02:15.703090 8602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a990961ebcc34988835efae4d61865b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5eb970024e7249fe80916cba830ee843" is_pre_election: true
I20260807 15:02:15.703133 9327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:02:15.703199 9327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 649e709be6fc4b46b52150eb1b3f2e6c)
I20260807 15:02:15.703231 9327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:02:15.704028 9327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } }
I20260807 15:02:15.704206 9327 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 election: Requested vote from peers 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413), a990961ebcc34988835efae4d61865b9 (127.1.127.252:40673)
I20260807 15:02:15.704223 8602 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a990961ebcc34988835efae4d61865b9 in current term 2: Already voted for candidate 5eb970024e7249fe80916cba830ee843 in this term.
I20260807 15:02:15.704419 8784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5eb970024e7249fe80916cba830ee843" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a990961ebcc34988835efae4d61865b9"
I20260807 15:02:15.704497 8784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:02:15.704882 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.704286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34272)
0807 15:02:15.704813 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.704853 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504}
I20260807 15:02:15.704995 8767 leader_election.cc:302] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a990961ebcc34988835efae4d61865b9; no voters: 5eb970024e7249fe80916cba830ee843, 649e709be6fc4b46b52150eb1b3f2e6c
I20260807 15:02:15.705295 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.705177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34276)
0807 15:02:15.705212 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.705264 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260807 15:02:15.705367 8574 leader_election.cc:334] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413): Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
I20260807 15:02:15.705405 8784 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eb970024e7249fe80916cba830ee843 in term 2.
I20260807 15:02:15.705453 9328 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:02:15.705559 8575 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5eb970024e7249fe80916cba830ee843, a990961ebcc34988835efae4d61865b9; no voters: 649e709be6fc4b46b52150eb1b3f2e6c
I20260807 15:02:15.705675 9327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:02:15.705829 9327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [term 2 LEADER]: Becoming Leader. State: Replica: 5eb970024e7249fe80916cba830ee843, State: Running, Role: LEADER
I20260807 15:02:15.705909 9327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } }
I20260807 15:02:15.706271 9337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb970024e7249fe80916cba830ee843. Latest consensus state: current_term: 2 leader_uuid: "5eb970024e7249fe80916cba830ee843" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:15.706386 9337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:15.706571 9339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:15.707154 9339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:15.707254 9339 catalog_manager.cc:1259] Loaded cluster ID: de00166f4a8940c38f29adc42af1d309
I20260807 15:02:15.707288 9339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:15.707469 9339 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:15.707567 9339 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Loaded TSK: 0
I20260807 15:02:15.707712 9339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:15.716766 8591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:15.718441 8591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:15.719667 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.719546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34290)
0807 15:02:15.719602 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.719638 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:02:15.719683 8784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [term 2 FOLLOWER]: Refusing update from remote peer 5eb970024e7249fe80916cba830ee843: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 15:02:15.719987 9337 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 15:02:15.720109 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.719998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.0.0.1:34298)
0807 15:02:15.720054 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.720086 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 15:02:15.720183 8574 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 -> Peer 649e709be6fc4b46b52150eb1b3f2e6c (127.1.127.254:41413): Couldn't send request to peer 649e709be6fc4b46b52150eb1b3f2e6c. Status: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:02:15.720736 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb970024e7249fe80916cba830ee843. Latest consensus state: current_term: 2 leader_uuid: "5eb970024e7249fe80916cba830ee843" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:15.720882 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:02:15.723423 9337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5eb970024e7249fe80916cba830ee843" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:15.723546 9337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:15.723621 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5eb970024e7249fe80916cba830ee843" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "649e709be6fc4b46b52150eb1b3f2e6c" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41413 } } peers { permanent_uuid: "5eb970024e7249fe80916cba830ee843" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 38271 } attrs { promote: false } } peers { permanent_uuid: "a990961ebcc34988835efae4d61865b9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 40673 } attrs { promote: false } } }
I20260807 15:02:15.723786 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a990961ebcc34988835efae4d61865b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:02:15.723629 9341 mvcc.cc:205] Tried to move back new op lower bound from 7315926776705363968 to 7315926776652025856. Current Snapshot: MvccSnapshot[applied={T|T < 7315926776705363968}]
I20260807 15:02:15.732358 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.727740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37837 (local address 127.0.0.1:54270)
0807 15:02:15.728971 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.728985 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.729048 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.729339 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.729340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.729342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.729391 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.729393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.730207 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.730208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.731142 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.731146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.731858 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.731859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.731930 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.731942 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.732188 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.732249 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1208}
I20260807 15:02:15.732661 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.727860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.1.127.194:37837)
0807 15:02:15.729205 (+ 1345us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.729207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.729214 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:15.729236 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.729237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.729252 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.729336 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.729456 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.730114 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.731275 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.731746 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.731855 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.731869 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.732498 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.732570 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1323,"thread_start_us":115,"threads_started":1}
I20260807 15:02:15.732475 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.727727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35511 (local address 127.0.0.1:57756)
0807 15:02:15.728121 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.728135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.728193 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.729020 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.729021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.729024 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.729086 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.729088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.729874 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.729875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.731267 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.731269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.732166 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.732167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.732235 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.732239 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.732269 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.732319 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366}
I20260807 15:02:15.733168 9045 tablet_service.cc:1467] Processing CreateTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 (DEFAULT_TABLE table=test-table [id=a901633ac9b74200a965726d63680d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:15.733345 9045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cebc2a3841d45088fd8fa63d21362b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:15.733425 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.728660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.1.127.193:35511)
0807 15:02:15.728857 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.728860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.728881 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:02:15.728916 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.728917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.728931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.729002 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.729148 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.729709 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.731406 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.732057 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.733218 (+ 1161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.733237 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.733261 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.733318 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
I20260807 15:02:15.733644 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.727727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38737 (local address 127.0.0.1:35134)
0807 15:02:15.729039 (+ 1312us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.729050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.729096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.729980 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.729981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.729982 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.730021 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.730022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.731114 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.731115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.732384 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.732386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.733396 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.733397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.733468 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.733474 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.733532 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.733578 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"mutex_wait_us":1101,"thread_start_us":87,"threads_started":1}
I20260807 15:02:15.733711 8898 tablet_service.cc:1467] Processing CreateTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 (DEFAULT_TABLE table=test-table [id=a901633ac9b74200a965726d63680d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:15.733810 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.728784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.1.127.195:38737)
0807 15:02:15.728906 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.728909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.729090 (+ 181us) server_negotiation.cc:407] Connection header received
0807 15:02:15.729850 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.729852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.729867 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.730087 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.730131 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.731003 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.732621 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.733304 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.733371 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.733384 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.733677 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.733730 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260807 15:02:15.733844 8898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cebc2a3841d45088fd8fa63d21362b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:15.735095 9349 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Bootstrap starting.
I20260807 15:02:15.735255 9186 tablet_service.cc:1467] Processing CreateTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 (DEFAULT_TABLE table=test-table [id=a901633ac9b74200a965726d63680d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:15.735404 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cebc2a3841d45088fd8fa63d21362b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:15.735512 9349 tablet_bootstrap.cc:654] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:15.735908 9350 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: Bootstrap starting.
I20260807 15:02:15.736339 9349 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: No bootstrap required, opened a new log
I20260807 15:02:15.736361 9350 tablet_bootstrap.cc:654] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:15.736413 9349 ts_tablet_manager.cc:1397] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:15.736596 9349 raft_consensus.cc:348] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.736680 9349 raft_consensus.cc:374] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:15.736701 9349 raft_consensus.cc:729] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca89db920c514d6794bfdec0c0b56014, State: Initialized, Role: FOLLOWER
I20260807 15:02:15.736719 9353 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Bootstrap starting.
I20260807 15:02:15.736768 9349 consensus_queue.cc:260] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.736944 9349 ts_tablet_manager.cc:1428] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:15.737004 9350 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: No bootstrap required, opened a new log
I20260807 15:02:15.737051 9350 ts_tablet_manager.cc:1397] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:15.737182 9350 raft_consensus.cc:348] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.737267 9350 raft_consensus.cc:374] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:15.737298 9350 raft_consensus.cc:729] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a766c4220014ba796fa0a75c7b08f53, State: Initialized, Role: FOLLOWER
I20260807 15:02:15.737485 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.737210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:42305)
0807 15:02:15.737384 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.737431 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260807 15:02:15.737614 9116 heartbeater.cc:503] Master 127.1.127.253:38271 was elected leader, sending a full tablet report...
I20260807 15:02:15.737883 9350 consensus_queue.cc:260] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.738116 9350 ts_tablet_manager.cc:1428] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:15.738502 8971 heartbeater.cc:503] Master 127.1.127.253:38271 was elected leader, sending a full tablet report...
I20260807 15:02:15.738643 9353 tablet_bootstrap.cc:654] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:15.739364 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.738270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:53497)
0807 15:02:15.739294 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.739332 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":88,"threads_started":1}
I20260807 15:02:15.739493 9353 tablet_bootstrap.cc:492] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: No bootstrap required, opened a new log
I20260807 15:02:15.739554 9353 ts_tablet_manager.cc:1397] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 15:02:15.739692 9353 raft_consensus.cc:348] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.739720 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.739618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:41309)
0807 15:02:15.739669 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.739696 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:02:15.739766 9353 raft_consensus.cc:374] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:15.739792 9353 raft_consensus.cc:729] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf09af3607c4e79a64d456d6d496c97, State: Initialized, Role: FOLLOWER
I20260807 15:02:15.739949 9353 consensus_queue.cc:260] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.740164 9353 ts_tablet_manager.cc:1428] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:15.740572 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.740353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:54347)
0807 15:02:15.740501 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.740538 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260807 15:02:15.740646 9257 heartbeater.cc:503] Master 127.1.127.253:38271 was elected leader, sending a full tablet report...
I20260807 15:02:15.741119 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.740816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:55527)
0807 15:02:15.740939 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.741075 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":85,"threads_started":1}
I20260807 15:02:15.947090 9359 raft_consensus.cc:482] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:15.947254 9359 raft_consensus.cc:504] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.947744 9359 leader_election.cc:288] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511), ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:15.952135 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.947933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37837 (local address 127.1.127.195:55239)
0807 15:02:15.947995 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.948009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.948079 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.948351 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.948352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.948355 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.948477 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.948479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.949504 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.949505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.950942 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.950946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.951799 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.951904 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.951907 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.951964 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.952015 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":132}
I20260807 15:02:15.952198 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.948133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55239 (local address 127.1.127.194:37837)
0807 15:02:15.948184 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.948186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.948195 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:15.948229 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.948230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.948243 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.948310 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.948600 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.949395 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951081 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.951668 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951754 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.951768 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.952061 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.952123 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:02:15.952261 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.948170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:44179 (local address 127.1.127.193:35511)
0807 15:02:15.948239 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:15.948242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:15.948252 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:15.948280 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:15.948281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:15.948292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:15.948354 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:15.948596 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.949367 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951081 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.951715 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951788 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.951799 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:15.952128 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:02:15.952186 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:02:15.952471 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.948037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35511 (local address 127.1.127.195:44179)
0807 15:02:15.948109 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.948121 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:15.948180 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:15.948364 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:15.948365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:15.948366 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:15.948486 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.948488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.949504 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.949505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.950942 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:15.950946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:15.951819 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:15.951820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:15.951894 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:15.951901 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:02:15.951981 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:02:15.952015 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":49}
I20260807 15:02:15.952582 9065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cebc2a3841d45088fd8fa63d21362b0" candidate_uuid: "6cf09af3607c4e79a64d456d6d496c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca89db920c514d6794bfdec0c0b56014" is_pre_election: true
I20260807 15:02:15.952700 9065 raft_consensus.cc:2454] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf09af3607c4e79a64d456d6d496c97 in term 0.
I20260807 15:02:15.952849 8919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cebc2a3841d45088fd8fa63d21362b0" candidate_uuid: "6cf09af3607c4e79a64d456d6d496c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a766c4220014ba796fa0a75c7b08f53" is_pre_election: true
I20260807 15:02:15.952893 9138 leader_election.cc:302] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cf09af3607c4e79a64d456d6d496c97, ca89db920c514d6794bfdec0c0b56014; no voters:
I20260807 15:02:15.953009 8919 raft_consensus.cc:2454] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf09af3607c4e79a64d456d6d496c97 in term 0.
I20260807 15:02:15.953092 9359 raft_consensus.cc:2781] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:15.953166 9359 raft_consensus.cc:482] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:15.953192 9359 raft_consensus.cc:3037] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:15.954113 9359 raft_consensus.cc:504] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.954340 9359 leader_election.cc:288] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [CANDIDATE]: Term 1 election: Requested vote from peers 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511), ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837)
I20260807 15:02:15.954560 8919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cebc2a3841d45088fd8fa63d21362b0" candidate_uuid: "6cf09af3607c4e79a64d456d6d496c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a766c4220014ba796fa0a75c7b08f53"
I20260807 15:02:15.954564 9065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cebc2a3841d45088fd8fa63d21362b0" candidate_uuid: "6cf09af3607c4e79a64d456d6d496c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca89db920c514d6794bfdec0c0b56014"
I20260807 15:02:15.954653 9065 raft_consensus.cc:3037] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:15.954653 8919 raft_consensus.cc:3037] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:15.955576 9065 raft_consensus.cc:2454] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf09af3607c4e79a64d456d6d496c97 in term 1.
I20260807 15:02:15.955739 8919 raft_consensus.cc:2454] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf09af3607c4e79a64d456d6d496c97 in term 1.
I20260807 15:02:15.955794 9138 leader_election.cc:302] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cf09af3607c4e79a64d456d6d496c97, ca89db920c514d6794bfdec0c0b56014; no voters:
I20260807 15:02:15.955978 9359 raft_consensus.cc:2781] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:15.956192 9359 raft_consensus.cc:686] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 1 LEADER]: Becoming Leader. State: Replica: 6cf09af3607c4e79a64d456d6d496c97, State: Running, Role: LEADER
I20260807 15:02:15.956288 9359 consensus_queue.cc:237] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } }
I20260807 15:02:15.956597 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:15.956477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:57507)
0807 15:02:15.956528 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:15.956563 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:02:15.957427 8591 catalog_manager.cc:5495] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "6cf09af3607c4e79a64d456d6d496c97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cf09af3607c4e79a64d456d6d496c97" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:16.024416 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.017716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.1.127.195:38737)
0807 15:02:16.017769 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:16.017771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:16.017780 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:16.017841 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:16.017842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:16.017862 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:16.018028 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:16.018075 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:16.018981 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:16.023168 (+ 4187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:16.023542 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:16.023650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:16.023896 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:16.024104 (+ 208us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:16.024212 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:16.024262 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:02:16.024289 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:02:16.025944 9065 raft_consensus.cc:1261] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Refusing update from remote peer 6cf09af3607c4e79a64d456d6d496c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:16.026043 8919 raft_consensus.cc:1261] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Refusing update from remote peer 6cf09af3607c4e79a64d456d6d496c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:16.026319 9359 consensus_queue.cc:1035] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca89db920c514d6794bfdec0c0b56014" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:16.026504 9362 consensus_queue.cc:1035] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a766c4220014ba796fa0a75c7b08f53" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:16.027384 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.027258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:45547)
0807 15:02:16.027303 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.027339 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:02:16.028558 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.028431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.194:53775)
0807 15:02:16.028489 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.028523 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 15:02:16.029122 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.029017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:40313)
0807 15:02:16.029059 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.029085 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:02:16.029599 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.029483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:41483)
0807 15:02:16.029534 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.029561 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:02:16.031737 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.031638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.195:34907)
0807 15:02:16.031678 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.031702 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 15:02:16.031821 8592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:16.031909 8592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:16.032210 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:16.032102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41413 (local address 127.1.127.193:41199)
0807 15:02:16.032147 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.032171 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:02:16.033048 9368 mvcc.cc:205] Tried to move back new op lower bound from 7315926777959723008 to 7315926777677979648. Current Snapshot: MvccSnapshot[applied={T|T < 7315926777959723008}]
I20260807 15:02:16.033521 8592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb970024e7249fe80916cba830ee843: Sending DeleteTablet for 3 replicas of tablet 7cebc2a3841d45088fd8fa63d21362b0
I20260807 15:02:16.033808 8898 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:16 UTC) from {username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:57756
I20260807 15:02:16.033963 9186 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:16 UTC) from {username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:35134
I20260807 15:02:16.034202 9379 tablet_replica.cc:330] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: stopping tablet replica
I20260807 15:02:16.034309 9379 raft_consensus.cc:2229] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:16.034288 9045 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cebc2a3841d45088fd8fa63d21362b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:16 UTC) from {username='oryx', principal='oryx/127.1.127.253@KRBTEST.COM'} at 127.0.0.1:54270
I20260807 15:02:16.034440 9379 raft_consensus.cc:2258] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:16.034538 9380 tablet_replica.cc:330] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: stopping tablet replica
I20260807 15:02:16.034607 9380 raft_consensus.cc:2229] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:16.034667 9380 raft_consensus.cc:2258] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:16.034768 9378 tablet_replica.cc:330] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53: stopping tablet replica
I20260807 15:02:16.034847 9378 raft_consensus.cc:2229] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:16.034904 9378 raft_consensus.cc:2258] T 7cebc2a3841d45088fd8fa63d21362b0 P 6a766c4220014ba796fa0a75c7b08f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:16.034976 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8743
I20260807 15:02:16.035066 9380 ts_tablet_manager.cc:1905] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:16.037992 9379 ts_tablet_manager.cc:1905] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:16.038349 9380 ts_tablet_manager.cc:1918] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:16.038425 9380 log.cc:1196] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/7cebc2a3841d45088fd8fa63d21362b0
I20260807 15:02:16.038767 9380 ts_tablet_manager.cc:1939] T 7cebc2a3841d45088fd8fa63d21362b0 P ca89db920c514d6794bfdec0c0b56014: Deleting consensus metadata
I20260807 15:02:16.039163 8574 catalog_manager.cc:4841] TS ca89db920c514d6794bfdec0c0b56014 (127.1.127.194:37837): tablet 7cebc2a3841d45088fd8fa63d21362b0 (table test-table [id=a901633ac9b74200a965726d63680d53]) successfully deleted
I20260807 15:02:16.040735 9379 ts_tablet_manager.cc:1918] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:16.040892 9379 log.cc:1196] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/7cebc2a3841d45088fd8fa63d21362b0
I20260807 15:02:16.041206 9379 ts_tablet_manager.cc:1939] T 7cebc2a3841d45088fd8fa63d21362b0 P 6cf09af3607c4e79a64d456d6d496c97: Deleting consensus metadata
I20260807 15:02:16.041721 8576 catalog_manager.cc:4841] TS 6cf09af3607c4e79a64d456d6d496c97 (127.1.127.195:38737): tablet 7cebc2a3841d45088fd8fa63d21362b0 (table test-table [id=a901633ac9b74200a965726d63680d53]) successfully deleted
W20260807 15:02:16.044791 9329 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:16.044348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35511 (local address 127.0.0.1:57772)
0807 15:02:16.044418 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:16.044430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:16.044478 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:16.044724 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:02:16.044952 8575 catalog_manager.cc:4584] TS 6a766c4220014ba796fa0a75c7b08f53 (127.1.127.193:35511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cebc2a3841d45088fd8fa63d21362b0: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:02:16.045172 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8977
I20260807 15:02:16.056507 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 9121
I20260807 15:02:16.064072 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8453
I20260807 15:02:16.071142 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 8646
2026-08-07T15:02:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5453 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:16 dist-test-slave-8tdl krb5kdc[9383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:16 dist-test-slave-8tdl krb5kdc[9383](info): set up 2 sockets
Aug 07 15:02:16 dist-test-slave-8tdl krb5kdc[9383](info): commencing operation
krb5kdc: starting...
W20260807 15:02:17.167230 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab.
Entry for principal oryx/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab.
I20260807 15:02:17.306528 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:34675
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab
--principal=oryx/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:17.392534 9397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:17.392819 9397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:17.392874 9397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:17.394160 9397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:17.394232 9397 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:02:17.394254 9397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:17.394276 9397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:17.394310 9397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:17.394335 9397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:17.396049 9397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:34675
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.254.keytab
--principal=oryx/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:17.396384 9397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:17.396647 9397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:17.401867 9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.401871 9402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.401867 9403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:17.402252 9397 server_base.cc:1034] running on GCE node
I20260807 15:02:17.402532 9397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:17.402928 9397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:17.404106 9397 hybrid_clock.cc:648] HybridClock initialized: now 1786114937404093 us; error 31 us; skew 500 ppm
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:17.406831 9397 init.cc:377] Logged in from keytab as oryx/127.1.127.254@KRBTEST.COM (short username oryx)
I20260807 15:02:17.407357 9397 webserver.cc:460] Webserver started at http://127.1.127.254:44205/ using document root <none> and password file <none>
I20260807 15:02:17.407616 9397 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:17.407663 9397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:17.407809 9397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:17.408978 9397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "506957b94b714f97bd01790155e8b0cd"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "613970076e2ad7cb2942c73aa53f8968"
server_key_iv: "f035e477ec9cfa01e84db0543fc43810"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.409397 9397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "506957b94b714f97bd01790155e8b0cd"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "613970076e2ad7cb2942c73aa53f8968"
server_key_iv: "f035e477ec9cfa01e84db0543fc43810"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.410982 9397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:17.411953 9412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.412209 9397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:17.412312 9397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "506957b94b714f97bd01790155e8b0cd"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "613970076e2ad7cb2942c73aa53f8968"
server_key_iv: "f035e477ec9cfa01e84db0543fc43810"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.412387 9397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:17.425076 9397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:17.427603 9397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:17.427731 9397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:17.433068 9397 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:34675
I20260807 15:02:17.433127 9475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:34675 every 8 connection(s)
I20260807 15:02:17.433509 9397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:17.434245 9476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:17.436820 9476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Bootstrap starting.
I20260807 15:02:17.437502 9476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:17.437827 9476 log.cc:824] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:17.438745 9476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: No bootstrap required, opened a new log
I20260807 15:02:17.440941 9476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } }
I20260807 15:02:17.441179 9476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:17.441241 9476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506957b94b714f97bd01790155e8b0cd, State: Initialized, Role: FOLLOWER
I20260807 15:02:17.441387 9476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } }
I20260807 15:02:17.441488 9476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:17.441552 9476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:17.441602 9476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:17.442476 9476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } }
I20260807 15:02:17.442612 9476 leader_election.cc:302] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506957b94b714f97bd01790155e8b0cd; no voters:
I20260807 15:02:17.442824 9476 leader_election.cc:288] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:17.443003 9479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:17.443171 9479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [term 1 LEADER]: Becoming Leader. State: Replica: 506957b94b714f97bd01790155e8b0cd, State: Running, Role: LEADER
I20260807 15:02:17.443143 9476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:17.443300 9479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } }
I20260807 15:02:17.443662 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 9397
I20260807 15:02:17.443646 9479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "506957b94b714f97bd01790155e8b0cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } } }
I20260807 15:02:17.443717 9479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:17.443790 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:17.443869 9479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 506957b94b714f97bd01790155e8b0cd. Latest consensus state: current_term: 1 leader_uuid: "506957b94b714f97bd01790155e8b0cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506957b94b714f97bd01790155e8b0cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34675 } } }
I20260807 15:02:17.443923 9479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:17.444092 1535 external_mini_cluster.cc:1428] Setting key 4b135a2d4400fde10368ed108f15a342
W20260807 15:02:17.445384 9495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:02:17.445493 9495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:02:17.445616 9485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:17.445837 9485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:17.447955 9485 catalog_manager.cc:1347] Generated new cluster ID: d5dfdb2036d447d29c79f03d0939744a
I20260807 15:02:17.448019 9485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:17.455847 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.444772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.1.127.254:34675)
0807 15:02:17.445123 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:17.445135 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:17.445179 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:02:17.445310 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:17.445318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:17.445379 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:17.445559 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:17.446379 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.447524 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.448555 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.448985 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.449078 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.451849 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:17.451858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:17.451859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:17.451872 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:17.453831 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.454319 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.454323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.454323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.454457 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.454710 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.454711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.454711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.455069 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:17.455220 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:17.455365 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 15:02:17.455613 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":111,"threads_started":1}
I20260807 15:02:17.472029 9485 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:17.472600 9485 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:17.477233 9485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Generated new TSK 0
I20260807 15:02:17.477437 9485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
I20260807 15:02:17.506129 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34675
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:17.594640 9502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:17.594928 9502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:17.594966 9502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:17.596257 9502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:17.596308 9502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:17.596330 9502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:02:17.596349 9502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:17.598012 9502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:17.598286 9502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:17.598549 9502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:17.603699 9510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.603699 9508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.603704 9507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:17.604051 9502 server_base.cc:1034] running on GCE node
I20260807 15:02:17.604303 9502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:17.604579 9502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:17.605747 9502 hybrid_clock.cc:648] HybridClock initialized: now 1786114937605723 us; error 34 us; skew 500 ppm
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:17.608522 9502 init.cc:377] Logged in from keytab as oryx/127.1.127.193@KRBTEST.COM (short username oryx)
I20260807 15:02:17.609053 9502 webserver.cc:460] Webserver started at http://127.1.127.193:45503/ using document root <none> and password file <none>
I20260807 15:02:17.609282 9502 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:17.609346 9502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:17.609499 9502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:17.610651 9502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "8b5ba5164a8c4333930af110c15db113"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "f13422e952d52f689fbf37d290a84c0f"
server_key_iv: "0c97be52801b9a00b722912a723aa8ee"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.611101 9502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "8b5ba5164a8c4333930af110c15db113"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "f13422e952d52f689fbf37d290a84c0f"
server_key_iv: "0c97be52801b9a00b722912a723aa8ee"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.612483 9502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:17.613435 9517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.613690 9502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:17.613802 9502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "8b5ba5164a8c4333930af110c15db113"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "f13422e952d52f689fbf37d290a84c0f"
server_key_iv: "0c97be52801b9a00b722912a723aa8ee"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.613884 9502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:17.636893 9502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:17.639482 9502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:17.639662 9502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:17.640339 9502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:17.640393 9502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.640429 9502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:17.640442 9502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:17.648547 9502 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:34599
I20260807 15:02:17.648656 9639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:34599 every 8 connection(s)
I20260807 15:02:17.648998 9502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:17.651764 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.640849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:35835 (local address 127.1.127.254:34675)
0807 15:02:17.640924 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:17.640928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:17.641213 (+ 285us) server_negotiation.cc:407] Connection header received
0807 15:02:17.641974 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:17.641975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:17.641991 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:17.642098 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:17.643079 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.643745 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.645149 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.645405 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.645493 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.648361 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:17.648363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:17.648363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:17.648372 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:17.650084 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.650606 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.650607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.650608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.650721 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.651017 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.651018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.651018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.651158 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:17.651241 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:17.651557 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:02:17.651662 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:02:17.651998 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.640820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.193:35835)
0807 15:02:17.641036 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:17.641089 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:17.641828 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:17.642147 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:17.642153 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:17.642593 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:17.642954 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.642962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.643879 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.643880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.645040 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.645044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.645522 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.645523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.645674 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.646297 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:17.646313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:17.648223 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:17.650192 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.650200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.650203 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.650514 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.650799 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.650800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.650800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.650934 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.651230 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:17.651238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:17.651400 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 15:02:17.651531 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 15:02:17.651715 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":123,"threads_started":1}
I20260807 15:02:17.652604 9640 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34675
I20260807 15:02:17.652698 9640 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:17.652902 9640 heartbeater.cc:511] Master 127.1.127.254:34675 requested a full tablet report, sending...
I20260807 15:02:17.652899 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 9502
I20260807 15:02:17.653023 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:17.653311 1535 external_mini_cluster.cc:1428] Setting key db1e08c378ff0542b5951df8ba826625
I20260807 15:02:17.653868 9429 ts_manager.cc:194] Registered new tserver with Master: 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599)
I20260807 15:02:17.654633 9429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:35835
I20260807 15:02:17.661005 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.653749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.193:47085)
0807 15:02:17.653892 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:17.653904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:17.653963 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:17.654140 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:17.654141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:17.654402 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:17.654484 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.654486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.655578 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.655579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.656505 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.656508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.656928 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.656929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.657003 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.657535 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:17.657541 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:17.657883 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:17.659978 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.659979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.659979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.660175 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.660401 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.660402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.660402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.660455 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.660752 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:17.660752 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:17.660791 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 15:02:17.660839 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:02:17.660922 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260807 15:02:17.661088 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.653887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:47085 (local address 127.1.127.254:34675)
0807 15:02:17.653933 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:17.653935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:17.653947 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:17.654057 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:17.654058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:17.654069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:17.654135 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:17.654595 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.655438 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.656601 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.656834 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.656926 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.657985 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:17.657987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:17.657987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:17.657995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:17.659874 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.660256 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.660258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.660259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.660327 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.660517 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.660518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.660518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.660687 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:17.660766 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:17.660877 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:02:17.660981 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
I20260807 15:02:17.687320 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34675
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:17.781916 9649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:17.782167 9649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:17.782195 9649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:17.783614 9649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:17.783668 9649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:17.783692 9649 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:02:17.783711 9649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:17.785426 9649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:17.785727 9649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:17.785975 9649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:17.791553 9657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.791553 9655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.791575 9654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:17.792025 9649 server_base.cc:1034] running on GCE node
I20260807 15:02:17.792284 9649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:17.792555 9649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:17.793720 9649 hybrid_clock.cc:648] HybridClock initialized: now 1786114937793695 us; error 38 us; skew 500 ppm
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:17.796703 9649 init.cc:377] Logged in from keytab as oryx/127.1.127.194@KRBTEST.COM (short username oryx)
I20260807 15:02:17.797278 9649 webserver.cc:460] Webserver started at http://127.1.127.194:33831/ using document root <none> and password file <none>
I20260807 15:02:17.797541 9649 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:17.797590 9649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:17.797721 9649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:17.798960 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "1fb26c0d50784fba94feedf7bda29ac9"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "92be7e8dc5a509742d9b75adf1190ffd"
server_key_iv: "fb9652ea50e6a9b27cdfef4083910bff"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.799337 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "1fb26c0d50784fba94feedf7bda29ac9"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "92be7e8dc5a509742d9b75adf1190ffd"
server_key_iv: "fb9652ea50e6a9b27cdfef4083910bff"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.800747 9649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:17.801592 9664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.801784 9649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:17.801888 9649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "1fb26c0d50784fba94feedf7bda29ac9"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "92be7e8dc5a509742d9b75adf1190ffd"
server_key_iv: "fb9652ea50e6a9b27cdfef4083910bff"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.801963 9649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:17.823673 9649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:17.826572 9649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:17.826795 9649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:17.827502 9649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:17.827561 9649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.827595 9649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:17.827616 9649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:17.835698 9649 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41433
I20260807 15:02:17.836165 9649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:17.836213 9784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41433 every 8 connection(s)
I20260807 15:02:17.838229 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.828016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:38455 (local address 127.1.127.254:34675)
0807 15:02:17.828091 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:17.828094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:17.828327 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:02:17.828951 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:17.828952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:17.828967 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:17.829092 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:17.830141 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.831033 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.832149 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.832515 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.832603 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.835150 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:17.835152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:17.835152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:17.835162 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:17.836894 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.837279 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.837281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.837281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.837340 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.837596 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.837597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.837597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.837709 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:17.837764 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:17.838089 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:02:17.838155 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:02:17.838560 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.827991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.194:38455)
0807 15:02:17.828180 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:17.828223 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:17.828821 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:17.829182 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:17.829186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:17.829671 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:17.829990 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.829997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.831179 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.831180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.832040 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.832043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.832640 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.832641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.832772 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.833476 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:17.833487 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:17.835030 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:17.836977 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.836982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.836983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.837204 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.837399 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.837400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.837400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.837538 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.837779 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:17.837785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:17.837905 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 15:02:17.838063 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 15:02:17.838199 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260807 15:02:17.839265 9785 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34675
I20260807 15:02:17.839377 9785 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:17.839584 9785 heartbeater.cc:511] Master 127.1.127.254:34675 requested a full tablet report, sending...
I20260807 15:02:17.840103 9428 ts_manager.cc:194] Registered new tserver with Master: 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433)
I20260807 15:02:17.840577 9428 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:38455
I20260807 15:02:17.844498 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 9649
I20260807 15:02:17.844663 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:17.844910 1535 external_mini_cluster.cc:1428] Setting key b89454a7ef8f235e07b15f87db3325d7
I20260807 15:02:17.848241 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.840476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57713 (local address 127.1.127.254:34675)
0807 15:02:17.840529 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:17.840531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:17.840659 (+ 128us) server_negotiation.cc:407] Connection header received
0807 15:02:17.840734 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:17.840735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:17.840746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:17.840801 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:17.841221 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.841862 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.842873 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.843078 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.843157 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.844208 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:17.844209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:17.844209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:17.844216 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:17.846603 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.847096 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.847097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.847097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.847159 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:17.847394 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:17.847395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:17.847396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:17.847585 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:17.847684 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:17.848041 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 15:02:17.848138 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:02:17.848258 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:17.840421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.194:57713)
0807 15:02:17.840591 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:17.840601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:17.840673 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:17.840807 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:17.840808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:17.841043 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:17.841124 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.841126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.842004 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.842005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.842794 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:17.842797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:17.843182 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:17.843184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:17.843243 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:17.843777 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:17.843783 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:17.844127 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:17.846737 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.846738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.846739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.847009 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.847230 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:17.847231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:17.847231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:17.847310 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:17.847705 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:17.847706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:17.847820 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:02:17.848009 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 15:02:17.848143 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
I20260807 15:02:17.879613 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34675
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:17.968003 9795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:17.968230 9795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:17.968257 9795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:17.969506 9795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:17.969555 9795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:17.969578 9795 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:02:17.969596 9795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:17.971649 9795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.127.254:34675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:17.971932 9795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:17.972201 9795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:17.977778 9800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.977831 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:17.977778 9805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:17.978351 9795 server_base.cc:1034] running on GCE node
I20260807 15:02:17.978587 9795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:17.978870 9795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:17.980041 9795 hybrid_clock.cc:648] HybridClock initialized: now 1786114937980021 us; error 29 us; skew 500 ppm
Aug 07 15:02:17 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:17.982939 9795 init.cc:377] Logged in from keytab as oryx/127.1.127.195@KRBTEST.COM (short username oryx)
I20260807 15:02:17.983446 9795 webserver.cc:460] Webserver started at http://127.1.127.195:33179/ using document root <none> and password file <none>
I20260807 15:02:17.983686 9795 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:17.983736 9795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:17.983904 9795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:17.985070 9795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "93f9cb6654d54820b23c1a344541ecad"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "a5a4792e28657523abae6e679817874d"
server_key_iv: "08c983fe96774e514e0bef7399deeefd"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.985443 9795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "93f9cb6654d54820b23c1a344541ecad"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "a5a4792e28657523abae6e679817874d"
server_key_iv: "08c983fe96774e514e0bef7399deeefd"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.986929 9795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:17.987984 9812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:17.988300 9795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:17.988451 9795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "93f9cb6654d54820b23c1a344541ecad"
format_stamp: "Formatted at 2026-08-07 15:02:17 on dist-test-slave-8tdl"
server_key: "a5a4792e28657523abae6e679817874d"
server_key_iv: "08c983fe96774e514e0bef7399deeefd"
server_key_version: "encryptionkey@0"
I20260807 15:02:17.988554 9795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:18.009366 9795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:18.012777 9795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:18.012987 9795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:18.013628 9795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:18.013690 9795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.013725 9795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:18.013754 9795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:18.021625 9795 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:44053
I20260807 15:02:18.021703 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:44053 every 8 connection(s)
I20260807 15:02:18.022018 9795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:18.024448 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.014129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:53599 (local address 127.1.127.254:34675)
0807 15:02:18.014204 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.014207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.014501 (+ 294us) server_negotiation.cc:407] Connection header received
0807 15:02:18.015353 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.015354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.015380 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.015454 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.016428 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.017017 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.018292 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.018498 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.018581 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.021376 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:18.021378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:18.021378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:18.021392 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:18.022974 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.023441 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.023442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.023443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.023506 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.023769 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.023770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.023770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.023894 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:18.023964 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.024252 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.024351 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:02:18.024657 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.014110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.195:53599)
0807 15:02:18.014328 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.014386 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.015205 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.015549 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.015554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.015999 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:18.016328 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.016335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.017112 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.017113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.018191 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.018194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.018609 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.018610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.018781 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.019362 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:18.019372 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:18.021266 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:18.023086 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.023094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.023097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.023360 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.023575 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.023576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.023576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.023694 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.023967 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:18.023975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:18.024118 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.024228 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.024399 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260807 15:02:18.025200 9933 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34675
I20260807 15:02:18.025300 9933 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:18.025545 9933 heartbeater.cc:511] Master 127.1.127.254:34675 requested a full tablet report, sending...
I20260807 15:02:18.025960 9428 ts_manager.cc:194] Registered new tserver with Master: 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.026352 9428 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:53599
I20260807 15:02:18.026564 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 9795
I20260807 15:02:18.026705 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:18.026930 1535 external_mini_cluster.cc:1428] Setting key 8f8e5304024f5f098184444db23dad67
I20260807 15:02:18.027700 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:02:18.032653 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.026577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:60731 (local address 127.1.127.254:34675)
0807 15:02:18.026614 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.026616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.026624 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:02:18.026734 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.026735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.026746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.026810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.027204 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.027771 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.028471 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.028678 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.028751 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.029768 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:18.029770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:18.029770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:18.029779 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:18.031392 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.031843 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.031844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.031845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.031904 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.032101 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.032101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.032102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.032209 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:18.032261 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.032439 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.032513 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:02:18.032693 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.026402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.1.127.195:60731)
0807 15:02:18.026541 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.026553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.026602 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.026816 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.026816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.027042 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:18.027123 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.027125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.027890 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.027891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.028385 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.028386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.028751 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.028752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.028826 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.029319 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:18.029324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:18.029651 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:18.031479 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.031480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.031480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.031762 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.031969 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.031970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.031971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.032041 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.032275 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:18.032276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:18.032333 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.032413 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.032551 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114938, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:02:18.052410 9496 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:18.046507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.1.127.254:34675)
0807 15:02:18.046575 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.046577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.046607 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:02:18.046632 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.046633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.046642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.046727 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.047408 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.047912 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.048756 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.048945 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.049006 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.052300 (+ 3294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53196: BlockingRecv error: recv got EOF from 127.0.0.1:53196 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114938, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:18.060681 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.053428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.1.127.254:34675)
0807 15:02:18.053475 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.053478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.053639 (+ 161us) server_negotiation.cc:407] Connection header received
0807 15:02:18.053675 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.053676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.053686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.053739 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.054225 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.054764 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.055521 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.055726 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.055794 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.058027 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:18.058029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:18.058029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:18.058036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:18.059536 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.059851 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.059852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.059852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.059975 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.060140 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.060141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.060141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.060259 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:18.060341 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.060453 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.060525 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:02:18.062551 9428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:18.064250 9428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:18.071208 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.066664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:44053 (local address 127.0.0.1:57210)
0807 15:02:18.066836 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.066884 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.066959 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.067180 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.067183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.067186 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.067266 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.067268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.068448 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.068449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.069761 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.069764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.070880 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.070881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.070977 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.070990 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.071053 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.071104 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260807 15:02:18.071516 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.066789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.1.127.195:44053)
0807 15:02:18.066990 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.066996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.067008 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:18.067055 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.067058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.067075 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.067166 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.067677 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.068376 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.069850 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.070655 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.070755 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.070845 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.071372 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.071422 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260807 15:02:18.072152 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.067195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34599 (local address 127.0.0.1:50346)
0807 15:02:18.067376 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.067385 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.067571 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.068663 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.068664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.068666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.068713 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.068716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.070017 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.070017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.070856 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.070859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.071910 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.071911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.071976 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.071981 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.072040 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.072080 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":45,"thread_start_us":90,"threads_started":1}
I20260807 15:02:18.072419 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.067515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.1.127.193:34599)
0807 15:02:18.068512 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.068517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.068525 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:02:18.068556 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.068558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.068571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.068662 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.068803 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.069931 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.070939 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.071823 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.071897 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.071968 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.072270 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.072314 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":87,"threads_started":1}
I20260807 15:02:18.072257 9865 tablet_service.cc:1467] Processing CreateTablet for tablet e6e614688e4846c48393cfdc790e3324 (DEFAULT_TABLE table=test-table [id=bfba2289611642b2b74afe69231d7bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.072662 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.067016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41433 (local address 127.0.0.1:57506)
0807 15:02:18.067182 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.067193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.067238 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.068962 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.068962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.068964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.069001 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.069003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.069999 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.070000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.071141 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.071143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.072409 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.072410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.072456 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.072459 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.072543 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.072586 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":28,"thread_start_us":89,"threads_started":1}
I20260807 15:02:18.072716 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.067500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57506 (local address 127.1.127.194:41433)
0807 15:02:18.068804 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.068808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.068817 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:18.068860 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.068861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.068873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.068952 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.069055 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.069890 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.071227 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.072225 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.072291 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.072368 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.072564 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.072604 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":98,"threads_started":1}
I20260807 15:02:18.072717 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e614688e4846c48393cfdc790e3324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.073371 9717 tablet_service.cc:1467] Processing CreateTablet for tablet e6e614688e4846c48393cfdc790e3324 (DEFAULT_TABLE table=test-table [id=bfba2289611642b2b74afe69231d7bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.073747 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e614688e4846c48393cfdc790e3324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.077099 9961 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Bootstrap starting.
I20260807 15:02:18.077652 9551 tablet_service.cc:1467] Processing CreateTablet for tablet e6e614688e4846c48393cfdc790e3324 (DEFAULT_TABLE table=test-table [id=bfba2289611642b2b74afe69231d7bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.077869 9961 tablet_bootstrap.cc:654] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.078053 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e614688e4846c48393cfdc790e3324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.078246 9961 log.cc:824] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:18.078254 9963 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Bootstrap starting.
I20260807 15:02:18.079093 9963 tablet_bootstrap.cc:654] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.079301 9961 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: No bootstrap required, opened a new log
I20260807 15:02:18.079384 9961 ts_tablet_manager.cc:1397] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:02:18.079464 9963 log.cc:824] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:18.080192 9963 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: No bootstrap required, opened a new log
I20260807 15:02:18.080273 9963 ts_tablet_manager.cc:1397] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.081696 9963 raft_consensus.cc:348] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.081722 9961 raft_consensus.cc:348] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.081863 9963 raft_consensus.cc:374] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.081910 9963 raft_consensus.cc:729] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.081913 9961 raft_consensus.cc:374] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.081954 9961 raft_consensus.cc:729] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.082078 9961 consensus_queue.cc:260] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.082036 9963 consensus_queue.cc:260] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.082383 9961 ts_tablet_manager.cc:1428] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:02:18.082391 9963 ts_tablet_manager.cc:1428] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:02:18.082371 9966 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Bootstrap starting.
I20260807 15:02:18.082518 9933 heartbeater.cc:503] Master 127.1.127.254:34675 was elected leader, sending a full tablet report...
I20260807 15:02:18.082847 9785 heartbeater.cc:503] Master 127.1.127.254:34675 was elected leader, sending a full tablet report...
I20260807 15:02:18.083611 9966 tablet_bootstrap.cc:654] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.083985 9966 log.cc:824] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:18.084757 9966 tablet_bootstrap.cc:492] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: No bootstrap required, opened a new log
I20260807 15:02:18.084893 9966 ts_tablet_manager.cc:1397] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:02:18.086319 9966 raft_consensus.cc:348] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.086483 9966 raft_consensus.cc:374] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.086509 9966 raft_consensus.cc:729] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5ba5164a8c4333930af110c15db113, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.086643 9966 consensus_queue.cc:260] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
W20260807 15:02:18.086827 9786 tablet.cc:2369] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:18.087021 9966 ts_tablet_manager.cc:1428] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:02:18.087069 9640 heartbeater.cc:503] Master 127.1.127.254:34675 was elected leader, sending a full tablet report...
W20260807 15:02:18.150944 9641 tablet.cc:2369] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:18.232090 9967 raft_consensus.cc:482] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:18.232244 9967 raft_consensus.cc:504] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.232664 9967 leader_election.cc:288] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053), 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599)
I20260807 15:02:18.236076 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.233325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:36437 (local address 127.1.127.193:34599)
0807 15:02:18.233375 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.233377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.233459 (+ 82us) server_negotiation.cc:407] Connection header received
0807 15:02:18.233508 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.233509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.233519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.233581 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.233732 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.234188 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235312 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.235693 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235777 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.235789 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.235940 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.235993 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:02:18.236105 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.233057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:39649 (local address 127.1.127.195:44053)
0807 15:02:18.233156 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.233158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.233234 (+ 76us) server_negotiation.cc:407] Connection header received
0807 15:02:18.233269 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.233270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.233282 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.233364 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.233585 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.234160 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235319 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.235698 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235788 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.235797 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.235992 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.236039 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:02:18.236114 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.233189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34599 (local address 127.1.127.194:36437)
0807 15:02:18.233390 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.233402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.233461 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.233592 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.233593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.233595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.233663 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.233664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.234291 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.234292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.235205 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.235208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.235781 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.235869 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.235872 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.235918 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.235968 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260807 15:02:18.236130 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.233039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:44053 (local address 127.1.127.194:39649)
0807 15:02:18.233097 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.233109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.233160 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.233396 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.233397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.233399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.233477 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.233479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.234268 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.234269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.235207 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.235210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.235799 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.235800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.235893 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.235896 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.235973 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.236017 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":22}
I20260807 15:02:18.236557 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e614688e4846c48393cfdc790e3324" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad" is_pre_election: true
I20260807 15:02:18.236552 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e614688e4846c48393cfdc790e3324" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113" is_pre_election: true
I20260807 15:02:18.236737 9886 raft_consensus.cc:2454] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.236739 9591 raft_consensus.cc:2454] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.237035 9667 leader_election.cc:302] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.237218 9967 raft_consensus.cc:2781] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:18.237349 9967 raft_consensus.cc:482] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:18.237375 9967 raft_consensus.cc:3037] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.238626 9967 raft_consensus.cc:504] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.238844 9967 leader_election.cc:288] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Requested vote from peers 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053), 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599)
I20260807 15:02:18.239058 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e614688e4846c48393cfdc790e3324" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113"
I20260807 15:02:18.239092 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e614688e4846c48393cfdc790e3324" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad"
I20260807 15:02:18.239174 9591 raft_consensus.cc:3037] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.239174 9886 raft_consensus.cc:3037] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.240012 9886 raft_consensus.cc:2454] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.240123 9591 raft_consensus.cc:2454] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.240187 9667 leader_election.cc:302] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.240289 9967 raft_consensus.cc:2781] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:18.240468 9967 raft_consensus.cc:686] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Running, Role: LEADER
I20260807 15:02:18.240554 9967 consensus_queue.cc:237] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } }
I20260807 15:02:18.241519 9429 catalog_manager.cc:5495] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "1fb26c0d50784fba94feedf7bda29ac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:18.255034 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.251797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.1.127.194:41433)
0807 15:02:18.251934 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.251936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.251947 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:18.251996 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.251997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.252017 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.252087 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:18.252309 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.253078 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.253977 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.254167 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.254226 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.254555 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.254664 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.254871 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.254891 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.254936 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 15:02:18.257016 9591 raft_consensus.cc:1261] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.257033 9886 raft_consensus.cc:1261] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.257418 9972 consensus_queue.cc:1035] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.257578 9967 consensus_queue.cc:1035] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.260581 9976 mvcc.cc:205] Tried to move back new op lower bound from 7315926787097714688 to 7315926787033919488. Current Snapshot: MvccSnapshot[applied={T|T < 7315926787097714688}]
I20260807 15:02:18.261253 9977 mvcc.cc:205] Tried to move back new op lower bound from 7315926787097714688 to 7315926787033919488. Current Snapshot: MvccSnapshot[applied={T|T < 7315926787097714688}]
I20260807 15:02:18.263244 9429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.263352 9429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.264228 9429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Sending DeleteTablet for 3 replicas of tablet e6e614688e4846c48393cfdc790e3324
I20260807 15:02:18.264650 9717 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e614688e4846c48393cfdc790e3324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57506
I20260807 15:02:18.264640 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e614688e4846c48393cfdc790e3324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57210
I20260807 15:02:18.264945 9986 tablet_replica.cc:330] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: stopping tablet replica
I20260807 15:02:18.264935 9551 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e614688e4846c48393cfdc790e3324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:50346
I20260807 15:02:18.265064 9986 raft_consensus.cc:2229] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.265113 9988 tablet_replica.cc:330] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: stopping tablet replica
I20260807 15:02:18.265209 9988 raft_consensus.cc:2229] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.265399 9988 raft_consensus.cc:2258] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.265424 9986 raft_consensus.cc:2258] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.265400 9429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:18.265547 9429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:18.265625 9987 tablet_replica.cc:330] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: stopping tablet replica
I20260807 15:02:18.265715 9987 raft_consensus.cc:2229] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:18.265805 9987 raft_consensus.cc:2258] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.266052 9986 ts_tablet_manager.cc:1905] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.266253 9987 ts_tablet_manager.cc:1905] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.267849 9551 tablet_service.cc:1467] Processing CreateTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 (DEFAULT_TABLE table=test-table [id=778364381c904d92a4df29f50e88a0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.267992 9987 ts_tablet_manager.cc:1918] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.268056 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b205794428248a3b1fbf020ad4d76e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.268087 9987 log.cc:1196] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/e6e614688e4846c48393cfdc790e3324
I20260807 15:02:18.268360 9987 ts_tablet_manager.cc:1939] T e6e614688e4846c48393cfdc790e3324 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting consensus metadata
I20260807 15:02:18.268476 9988 ts_tablet_manager.cc:1905] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.268796 9415 catalog_manager.cc:4841] TS 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433): tablet e6e614688e4846c48393cfdc790e3324 (table test-table [id=bfba2289611642b2b74afe69231d7bfa]) successfully deleted
I20260807 15:02:18.269516 9966 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Bootstrap starting.
I20260807 15:02:18.269574 9717 tablet_service.cc:1467] Processing CreateTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 (DEFAULT_TABLE table=test-table [id=778364381c904d92a4df29f50e88a0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.269688 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b205794428248a3b1fbf020ad4d76e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.269958 9966 tablet_bootstrap.cc:654] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.270506 9988 ts_tablet_manager.cc:1918] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.270577 9966 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: No bootstrap required, opened a new log
I20260807 15:02:18.270666 9966 ts_tablet_manager.cc:1397] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:18.270723 9988 log.cc:1196] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/e6e614688e4846c48393cfdc790e3324
I20260807 15:02:18.270870 9966 raft_consensus.cc:348] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.270968 9966 raft_consensus.cc:374] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.270996 9966 raft_consensus.cc:729] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5ba5164a8c4333930af110c15db113, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.270968 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 (DEFAULT_TABLE table=test-table [id=778364381c904d92a4df29f50e88a0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.271042 9966 consensus_queue.cc:260] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.271112 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b205794428248a3b1fbf020ad4d76e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.271158 9966 ts_tablet_manager.cc:1428] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.271741 9988 ts_tablet_manager.cc:1939] T e6e614688e4846c48393cfdc790e3324 P 8b5ba5164a8c4333930af110c15db113: Deleting consensus metadata
I20260807 15:02:18.272279 9415 catalog_manager.cc:4841] TS 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599): tablet e6e614688e4846c48393cfdc790e3324 (table test-table [id=bfba2289611642b2b74afe69231d7bfa]) successfully deleted
I20260807 15:02:18.272647 9961 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Bootstrap starting.
I20260807 15:02:18.273077 9961 tablet_bootstrap.cc:654] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.273077 9963 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Bootstrap starting.
I20260807 15:02:18.273430 9986 ts_tablet_manager.cc:1918] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.273524 9986 log.cc:1196] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/e6e614688e4846c48393cfdc790e3324
I20260807 15:02:18.273535 9963 tablet_bootstrap.cc:654] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.273823 9986 ts_tablet_manager.cc:1939] T e6e614688e4846c48393cfdc790e3324 P 93f9cb6654d54820b23c1a344541ecad: Deleting consensus metadata
I20260807 15:02:18.273988 9961 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: No bootstrap required, opened a new log
I20260807 15:02:18.274060 9961 ts_tablet_manager.cc:1397] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:18.274222 9961 raft_consensus.cc:348] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.274299 9961 raft_consensus.cc:374] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.274322 9961 raft_consensus.cc:729] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.274336 9415 catalog_manager.cc:4841] TS 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053): tablet e6e614688e4846c48393cfdc790e3324 (table test-table [id=bfba2289611642b2b74afe69231d7bfa]) successfully deleted
I20260807 15:02:18.274363 9961 consensus_queue.cc:260] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.274456 9961 ts_tablet_manager.cc:1428] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.275048 9963 tablet_bootstrap.cc:492] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: No bootstrap required, opened a new log
I20260807 15:02:18.275106 9963 ts_tablet_manager.cc:1397] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.275262 9963 raft_consensus.cc:348] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.275336 9963 raft_consensus.cc:374] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.275357 9963 raft_consensus.cc:729] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.275398 9963 consensus_queue.cc:260] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.275487 9963 ts_tablet_manager.cc:1428] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.314102 9982 raft_consensus.cc:482] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:18.314255 9982 raft_consensus.cc:504] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.314471 9982 leader_election.cc:288] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.314786 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b205794428248a3b1fbf020ad4d76e9" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad" is_pre_election: true
I20260807 15:02:18.314786 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b205794428248a3b1fbf020ad4d76e9" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113" is_pre_election: true
I20260807 15:02:18.314905 9591 raft_consensus.cc:2454] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.314905 9886 raft_consensus.cc:2454] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.315192 9667 leader_election.cc:302] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 8b5ba5164a8c4333930af110c15db113; no voters:
I20260807 15:02:18.315335 9982 raft_consensus.cc:2781] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:18.315377 9982 raft_consensus.cc:482] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:18.315399 9982 raft_consensus.cc:3037] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.316465 9982 raft_consensus.cc:504] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.316601 9982 leader_election.cc:288] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.316800 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b205794428248a3b1fbf020ad4d76e9" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113"
I20260807 15:02:18.316841 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b205794428248a3b1fbf020ad4d76e9" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad"
I20260807 15:02:18.316890 9591 raft_consensus.cc:3037] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.316912 9886 raft_consensus.cc:3037] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.317690 9886 raft_consensus.cc:2454] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.317690 9591 raft_consensus.cc:2454] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.317865 9667 leader_election.cc:302] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 8b5ba5164a8c4333930af110c15db113; no voters:
I20260807 15:02:18.318003 9982 raft_consensus.cc:2781] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:18.318076 9982 raft_consensus.cc:686] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Running, Role: LEADER
I20260807 15:02:18.318137 9982 consensus_queue.cc:237] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.318753 9428 catalog_manager.cc:5495] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "1fb26c0d50784fba94feedf7bda29ac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114937, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.254@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:18.352306 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.343611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34675 (local address 127.0.0.1:53212)
0807 15:02:18.343832 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.343845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.343924 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.344233 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.344234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.344579 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:18.344635 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.344637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.345594 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.345595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.346120 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.346122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.346615 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.346616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.346712 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.347370 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:18.347375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:18.348834 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:18.350827 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.350837 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.350839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.351105 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.351400 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:18.351401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:18.351401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:18.351470 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:18.351796 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:18.351804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:18.351954 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.352095 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.352211 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":118,"threads_started":1}
I20260807 15:02:18.352408 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.343681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.1.127.254:34675)
0807 15:02:18.343784 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.343786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.343918 (+ 132us) server_negotiation.cc:407] Connection header received
0807 15:02:18.344108 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.344109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.344124 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.344191 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.344729 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.345492 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.346218 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.346499 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.346583 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.348951 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:18.348953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:18.348953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:18.348961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:18.350730 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.351215 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.351216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.351217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.351296 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.351565 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.351566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.351566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.351719 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:18.351800 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.352148 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.352285 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":47}
I20260807 15:02:18.355298 9428 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:53212:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:02:18.355540 9428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:18.357086 9551 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbbd8ff643448c0ad46f9015a762b4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d2576ead54cb2b8ff1ca4887ac772]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:02:18.357147 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbbd8ff643448c0ad46f9015a762b4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d2576ead54cb2b8ff1ca4887ac772]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:02:18.357275 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbbd8ff643448c0ad46f9015a762b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.357275 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbbd8ff643448c0ad46f9015a762b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.358927 9961 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: Bootstrap starting.
I20260807 15:02:18.359009 9717 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbbd8ff643448c0ad46f9015a762b4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b57d2576ead54cb2b8ff1ca4887ac772]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:02:18.359163 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbbd8ff643448c0ad46f9015a762b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.359448 9961 tablet_bootstrap.cc:654] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.359903 9966 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: Bootstrap starting.
I20260807 15:02:18.360353 9966 tablet_bootstrap.cc:654] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.360390 9963 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: Bootstrap starting.
I20260807 15:02:18.360791 9963 tablet_bootstrap.cc:654] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.361029 9961 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: No bootstrap required, opened a new log
I20260807 15:02:18.361078 9961 ts_tablet_manager.cc:1397] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.361234 9961 raft_consensus.cc:348] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.361310 9961 raft_consensus.cc:374] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.361330 9961 raft_consensus.cc:729] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.361388 9961 consensus_queue.cc:260] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.361433 9963 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: No bootstrap required, opened a new log
I20260807 15:02:18.361478 9961 ts_tablet_manager.cc:1428] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.361479 9963 ts_tablet_manager.cc:1397] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:18.361575 9968 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.361627 9963 raft_consensus.cc:348] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.361673 9968 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.361694 9963 raft_consensus.cc:374] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.361717 9963 raft_consensus.cc:729] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.361759 9963 consensus_queue.cc:260] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.361862 9963 ts_tablet_manager.cc:1428] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:02:18.361984 9982 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.362056 9982 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.362159 9966 tablet_bootstrap.cc:492] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: No bootstrap required, opened a new log
I20260807 15:02:18.362217 9966 ts_tablet_manager.cc:1397] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:02:18.362378 9966 raft_consensus.cc:348] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.362450 9966 raft_consensus.cc:374] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.362468 9966 raft_consensus.cc:729] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5ba5164a8c4333930af110c15db113, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.362519 9966 consensus_queue.cc:260] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.362613 9966 ts_tablet_manager.cc:1428] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:02:18.362666 9970 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.362833 9970 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.518136 9968 raft_consensus.cc:482] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:18.518272 9968 raft_consensus.cc:504] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.518637 9968 leader_election.cc:288] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433)
I20260807 15:02:18.521600 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.518945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:40215 (local address 127.1.127.193:34599)
0807 15:02:18.519006 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.519007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.519063 (+ 56us) server_negotiation.cc:407] Connection header received
0807 15:02:18.519096 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.519097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.519106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.519173 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.519374 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.519879 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.520855 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.521223 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.521316 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.521325 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.521484 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.521526 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:02:18.521600 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.518878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34599 (local address 127.1.127.195:40215)
0807 15:02:18.518935 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.518948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.518997 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.519195 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.519197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.519200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.519284 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.519286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.519966 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.519967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.520750 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.520752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.521315 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.521316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.521395 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.521398 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.521470 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.521524 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:02:18.521888 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbbd8ff643448c0ad46f9015a762b4a" candidate_uuid: "93f9cb6654d54820b23c1a344541ecad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113" is_pre_election: true
I20260807 15:02:18.522015 9591 raft_consensus.cc:2454] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f9cb6654d54820b23c1a344541ecad in term 0.
I20260807 15:02:18.522243 9815 leader_election.cc:302] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5ba5164a8c4333930af110c15db113, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.522323 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.519245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55601 (local address 127.1.127.194:41433)
0807 15:02:18.519319 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.519321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.519328 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:02:18.519353 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.519354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.519365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.519428 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:18.519615 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.520079 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.521406 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.521901 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.521972 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.521979 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.522211 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.522260 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:02:18.522321 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.519052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41433 (local address 127.1.127.195:55601)
0807 15:02:18.519181 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.519194 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.519243 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.519433 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.519434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.519435 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:18.519524 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.519525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.520158 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.520158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.521308 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.521310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.521994 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.522048 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.522132 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.522137 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.522194 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.522232 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260807 15:02:18.522473 9968 raft_consensus.cc:2781] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:18.522532 9968 raft_consensus.cc:482] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:18.522565 9968 raft_consensus.cc:3037] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.522565 9739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbbd8ff643448c0ad46f9015a762b4a" candidate_uuid: "93f9cb6654d54820b23c1a344541ecad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fb26c0d50784fba94feedf7bda29ac9" is_pre_election: true
I20260807 15:02:18.522749 9739 raft_consensus.cc:2454] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f9cb6654d54820b23c1a344541ecad in term 0.
I20260807 15:02:18.523382 9968 raft_consensus.cc:504] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
W20260807 15:02:18.523429 9934 tablet.cc:2369] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:18.523531 9968 leader_election.cc:288] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [CANDIDATE]: Term 1 election: Requested vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433)
I20260807 15:02:18.523732 9739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbbd8ff643448c0ad46f9015a762b4a" candidate_uuid: "93f9cb6654d54820b23c1a344541ecad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fb26c0d50784fba94feedf7bda29ac9"
I20260807 15:02:18.523759 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbbd8ff643448c0ad46f9015a762b4a" candidate_uuid: "93f9cb6654d54820b23c1a344541ecad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113"
I20260807 15:02:18.523826 9739 raft_consensus.cc:3037] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.523833 9591 raft_consensus.cc:3037] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.524456 9591 raft_consensus.cc:2454] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f9cb6654d54820b23c1a344541ecad in term 1.
I20260807 15:02:18.524627 9815 leader_election.cc:302] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5ba5164a8c4333930af110c15db113, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.524648 9739 raft_consensus.cc:2454] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f9cb6654d54820b23c1a344541ecad in term 1.
I20260807 15:02:18.524712 9968 raft_consensus.cc:2781] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:18.524896 9968 raft_consensus.cc:686] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [term 1 LEADER]: Becoming Leader. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Running, Role: LEADER
I20260807 15:02:18.524974 9968 consensus_queue.cc:237] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.525363 10004 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: TxnStatusTablet state changed. Reason: New leader 93f9cb6654d54820b23c1a344541ecad. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.525456 10004 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:02:18.525684 10006 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:02:18.525815 10006 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:02:18.525931 9429 catalog_manager.cc:5495] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad reported cstate change: term changed from 0 to 1, leader changed from <none> to 93f9cb6654d54820b23c1a344541ecad (127.1.127.195). New cstate: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } health_report { overall_health: HEALTHY } } }
I20260807 15:02:18.551429 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.548175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.1.127.195:44053)
0807 15:02:18.548236 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.548238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.548253 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:02:18.548340 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.548341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.548366 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.548431 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:18.548607 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.549164 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.550404 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.550626 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.550783 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.551051 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.551165 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.551246 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.551332 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.551362 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:02:18.551491 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.548064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:44053 (local address 127.0.0.1:57212)
0807 15:02:18.548160 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.548175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.548241 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.548461 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.548462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.548464 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:02:18.548531 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.548534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.549253 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.549254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.550295 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.550299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.550793 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.550794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.550877 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.550903 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.551250 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.551262 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.551326 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.551384 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:02:18.553267 9591 raft_consensus.cc:1261] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Refusing update from remote peer 93f9cb6654d54820b23c1a344541ecad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.553359 9739 raft_consensus.cc:1261] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Refusing update from remote peer 93f9cb6654d54820b23c1a344541ecad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.553791 10004 consensus_queue.cc:1035] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.553990 9968 consensus_queue.cc:1035] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.554827 9970 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: TxnStatusTablet state changed. Reason: New leader 93f9cb6654d54820b23c1a344541ecad. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.554939 9970 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.554955 9982 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: TxnStatusTablet state changed. Reason: New leader 93f9cb6654d54820b23c1a344541ecad. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.555048 9982 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.555543 10004 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.555644 10004 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:02:18.556347 9968 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.556448 9968 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 93f9cb6654d54820b23c1a344541ecad: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:02:18.556682 9970 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.556787 9970 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 8b5ba5164a8c4333930af110c15db113: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.556840 9982 tablet_replica.cc:439] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f9cb6654d54820b23c1a344541ecad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } } }
I20260807 15:02:18.557086 9982 tablet_replica.cc:442] T 9fbbd8ff643448c0ad46f9015a762b4a P 1fb26c0d50784fba94feedf7bda29ac9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:02:18.557152 9975 mvcc.cc:205] Tried to move back new op lower bound from 7315926788311326720 to 7315926788198920192. Current Snapshot: MvccSnapshot[applied={T|T < 7315926788311326720 or (T in {7315926788311326720})}]
I20260807 15:02:18.562757 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.559639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:55393 (local address 127.1.127.195:44053)
0807 15:02:18.559703 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.559705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.559896 (+ 191us) server_negotiation.cc:407] Connection header received
0807 15:02:18.559950 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.559952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.559964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.560037 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:18.560176 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.561007 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.561887 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.562229 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.562309 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.562485 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.562525 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.562583 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.562632 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.562654 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:02:18.562812 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.559570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:44053 (local address 127.1.127.194:55393)
0807 15:02:18.559817 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.559829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.559902 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.560039 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.560040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.560042 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:02:18.560088 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.560089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.561106 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.561107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.561789 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.561792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.562327 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.562328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.562379 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.562395 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.562589 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.562591 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.562626 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.562668 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":174,"threads_started":1}
I20260807 15:02:18.565636 9591 raft_consensus.cc:1261] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.565670 9886 raft_consensus.cc:1261] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.566008 9982 consensus_queue.cc:1035] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.566151 9972 consensus_queue.cc:1035] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.575317 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.572051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:35473 (local address 127.1.127.194:41433)
0807 15:02:18.572097 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.572099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.572246 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:02:18.572372 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.572373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.572385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.572427 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:18.572684 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.573394 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.574593 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.574854 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.574911 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.575112 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.575143 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.575180 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.575234 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.575256 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:02:18.575392 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.571877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41433 (local address 127.1.127.195:35473)
0807 15:02:18.572182 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.572194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:18.572262 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:18.572501 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:18.572502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:18.572504 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:02:18.572590 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.572591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.573506 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.573507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.574490 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:18.574493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.574925 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.574926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:18.574998 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.575022 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.575181 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.575183 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:02:18.575231 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:02:18.575279 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":75,"threads_started":1}
I20260807 15:02:18.584282 9428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.584422 9428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.585093 9428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Sending DeleteTablet for 3 replicas of tablet 9b205794428248a3b1fbf020ad4d76e9
I20260807 15:02:18.585354 9551 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:50346
I20260807 15:02:18.585377 9717 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57506
I20260807 15:02:18.585462 9988 tablet_replica.cc:330] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: stopping tablet replica
I20260807 15:02:18.585520 9988 raft_consensus.cc:2229] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.585587 9987 tablet_replica.cc:330] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: stopping tablet replica
I20260807 15:02:18.585618 9988 raft_consensus.cc:2258] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.585655 9987 raft_consensus.cc:2229] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:18.585731 9987 raft_consensus.cc:2258] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.586144 9988 ts_tablet_manager.cc:1905] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.586195 9987 ts_tablet_manager.cc:1905] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.587908 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b205794428248a3b1fbf020ad4d76e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57210
I20260807 15:02:18.588244 9987 ts_tablet_manager.cc:1918] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:02:18.588243 9988 ts_tablet_manager.cc:1918] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:02:18.588299 9987 log.cc:1196] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/9b205794428248a3b1fbf020ad4d76e9
I20260807 15:02:18.588299 9988 log.cc:1196] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/9b205794428248a3b1fbf020ad4d76e9
I20260807 15:02:18.588534 9988 ts_tablet_manager.cc:1939] T 9b205794428248a3b1fbf020ad4d76e9 P 8b5ba5164a8c4333930af110c15db113: Deleting consensus metadata
I20260807 15:02:18.588591 9987 ts_tablet_manager.cc:1939] T 9b205794428248a3b1fbf020ad4d76e9 P 1fb26c0d50784fba94feedf7bda29ac9: Deleting consensus metadata
I20260807 15:02:18.589196 9415 catalog_manager.cc:4841] TS 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433): tablet 9b205794428248a3b1fbf020ad4d76e9 (table test-table [id=778364381c904d92a4df29f50e88a0e1]) successfully deleted
I20260807 15:02:18.589535 9415 catalog_manager.cc:4841] TS 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599): tablet 9b205794428248a3b1fbf020ad4d76e9 (table test-table [id=778364381c904d92a4df29f50e88a0e1]) successfully deleted
I20260807 15:02:18.589614 9986 tablet_replica.cc:330] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: stopping tablet replica
I20260807 15:02:18.589691 9986 raft_consensus.cc:2229] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.589751 9986 raft_consensus.cc:2258] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.590116 9986 ts_tablet_manager.cc:1905] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.592151 9986 ts_tablet_manager.cc:1918] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:02:18.592212 9986 log.cc:1196] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/9b205794428248a3b1fbf020ad4d76e9
I20260807 15:02:18.592451 9986 ts_tablet_manager.cc:1939] T 9b205794428248a3b1fbf020ad4d76e9 P 93f9cb6654d54820b23c1a344541ecad: Deleting consensus metadata
I20260807 15:02:18.593518 9415 catalog_manager.cc:4841] TS 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053): tablet 9b205794428248a3b1fbf020ad4d76e9 (table test-table [id=778364381c904d92a4df29f50e88a0e1]) successfully deleted
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:02:18.613433 9496 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:18.606179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.1.127.254:34675)
0807 15:02:18.606246 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.606248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.606345 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:02:18.606402 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.606404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.606416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.606497 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.607362 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.608124 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.609059 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.609391 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.609465 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.613348 (+ 3883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53214: BlockingRecv error: recv got EOF from 127.0.0.1:53214 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 15:02:18 dist-test-slave-8tdl krb5kdc[9383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM
I20260807 15:02:18.623937 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.614609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.1.127.254:34675)
0807 15:02:18.614720 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.614722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.614734 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:18.614843 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.614844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.614856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.614933 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:18.615496 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.616263 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.617223 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.617518 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.617631 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.620561 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:18.620562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:18.620563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:18.620572 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:18.622351 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.622966 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.622967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.622968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.623041 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:18.623287 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:18.623288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:18.623289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:18.623454 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:18.623564 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.623737 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.623831 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:02:18.625991 9428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:18.626273 9428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:18.627755 9551 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d (DEFAULT_TABLE table=test-table [id=57d280e9ea484d31881c4b5f875cc9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.627789 9717 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d (DEFAULT_TABLE table=test-table [id=57d280e9ea484d31881c4b5f875cc9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.627863 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d (DEFAULT_TABLE table=test-table [id=57d280e9ea484d31881c4b5f875cc9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.627919 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6da0ba2d78499f9344b4baacbf6b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.627919 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6da0ba2d78499f9344b4baacbf6b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.628072 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6da0ba2d78499f9344b4baacbf6b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.629314 9966 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Bootstrap starting.
I20260807 15:02:18.629314 9963 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Bootstrap starting.
I20260807 15:02:18.629571 9961 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Bootstrap starting.
I20260807 15:02:18.629678 9963 tablet_bootstrap.cc:654] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.629760 9966 tablet_bootstrap.cc:654] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.630012 9961 tablet_bootstrap.cc:654] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.630482 9966 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: No bootstrap required, opened a new log
I20260807 15:02:18.630556 9966 ts_tablet_manager.cc:1397] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:02:18.630755 9963 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: No bootstrap required, opened a new log
I20260807 15:02:18.630805 9963 ts_tablet_manager.cc:1397] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.630781 9966 raft_consensus.cc:348] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.630869 9966 raft_consensus.cc:374] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.630896 9966 raft_consensus.cc:729] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5ba5164a8c4333930af110c15db113, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.630939 9966 consensus_queue.cc:260] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.630920 9963 raft_consensus.cc:348] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.630983 9963 raft_consensus.cc:374] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.631004 9963 raft_consensus.cc:729] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.631028 9966 ts_tablet_manager.cc:1428] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:02:18.631059 9963 consensus_queue.cc:260] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.631155 9963 ts_tablet_manager.cc:1428] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.631603 9961 tablet_bootstrap.cc:492] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: No bootstrap required, opened a new log
I20260807 15:02:18.631659 9961 ts_tablet_manager.cc:1397] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.631830 9961 raft_consensus.cc:348] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.631899 9961 raft_consensus.cc:374] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.631922 9961 raft_consensus.cc:729] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.631981 9961 consensus_queue.cc:260] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.632084 9961 ts_tablet_manager.cc:1428] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:02:18.739853 9982 raft_consensus.cc:482] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:18.739965 9982 raft_consensus.cc:504] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.740108 9982 leader_election.cc:288] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.740458 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6da0ba2d78499f9344b4baacbf6b5d" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad" is_pre_election: true
I20260807 15:02:18.740456 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6da0ba2d78499f9344b4baacbf6b5d" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113" is_pre_election: true
I20260807 15:02:18.740578 9886 raft_consensus.cc:2454] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.740578 9591 raft_consensus.cc:2454] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.740754 9667 leader_election.cc:302] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.740963 9982 raft_consensus.cc:2781] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:18.741027 9982 raft_consensus.cc:482] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:18.741060 9982 raft_consensus.cc:3037] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.741829 9982 raft_consensus.cc:504] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.741983 9982 leader_election.cc:288] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.742254 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6da0ba2d78499f9344b4baacbf6b5d" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113"
I20260807 15:02:18.742257 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6da0ba2d78499f9344b4baacbf6b5d" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad"
I20260807 15:02:18.742350 9886 raft_consensus.cc:3037] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.742357 9591 raft_consensus.cc:3037] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.743196 9591 raft_consensus.cc:2454] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.743409 9886 raft_consensus.cc:2454] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.743443 9667 leader_election.cc:302] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 8b5ba5164a8c4333930af110c15db113; no voters:
I20260807 15:02:18.743539 9982 raft_consensus.cc:2781] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:18.743584 9982 raft_consensus.cc:686] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Running, Role: LEADER
I20260807 15:02:18.743637 9982 consensus_queue.cc:237] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.744241 9428 catalog_manager.cc:5495] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "1fb26c0d50784fba94feedf7bda29ac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:18.757795 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.755436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.1.127.194:41433)
0807 15:02:18.755510 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:18.755512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:18.755524 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:18.755554 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:18.755555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:18.755580 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:18.755633 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:18.755802 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.756327 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.757167 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:18.757368 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:18.757432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:18.757586 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:18.757617 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:18.757657 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:18.757708 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:02:18.757731 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:02:18.759027 9591 raft_consensus.cc:1261] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.759135 9886 raft_consensus.cc:1261] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.759398 9982 consensus_queue.cc:1035] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.759514 9972 consensus_queue.cc:1035] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.762113 9428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.762197 9428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.762920 9428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Sending DeleteTablet for 3 replicas of tablet 6a6da0ba2d78499f9344b4baacbf6b5d
I20260807 15:02:18.763188 9551 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:50346
I20260807 15:02:18.763228 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57210
I20260807 15:02:18.763269 9717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6da0ba2d78499f9344b4baacbf6b5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57506
I20260807 15:02:18.763293 9988 tablet_replica.cc:330] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: stopping tablet replica
I20260807 15:02:18.763381 9987 tablet_replica.cc:330] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: stopping tablet replica
I20260807 15:02:18.763375 9986 tablet_replica.cc:330] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: stopping tablet replica
I20260807 15:02:18.763423 9988 raft_consensus.cc:2229] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.763433 9987 raft_consensus.cc:2229] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:18.763449 9986 raft_consensus.cc:2229] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.763482 9987 raft_consensus.cc:2258] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.763478 9988 raft_consensus.cc:2258] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.763492 9986 raft_consensus.cc:2258] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.763851 9987 ts_tablet_manager.cc:1905] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.763779 9428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:02:18.763873 9986 ts_tablet_manager.cc:1905] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.763878 9988 ts_tablet_manager.cc:1905] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:02:18.763934 9428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:18.766000 9987 ts_tablet_manager.cc:1918] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.765969 9988 ts_tablet_manager.cc:1918] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.766070 9988 log.cc:1196] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/6a6da0ba2d78499f9344b4baacbf6b5d
I20260807 15:02:18.766070 9987 log.cc:1196] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/6a6da0ba2d78499f9344b4baacbf6b5d
I20260807 15:02:18.766326 9988 ts_tablet_manager.cc:1939] T 6a6da0ba2d78499f9344b4baacbf6b5d P 8b5ba5164a8c4333930af110c15db113: Deleting consensus metadata
I20260807 15:02:18.766357 9987 ts_tablet_manager.cc:1939] T 6a6da0ba2d78499f9344b4baacbf6b5d P 1fb26c0d50784fba94feedf7bda29ac9: Deleting consensus metadata
I20260807 15:02:18.766654 9986 ts_tablet_manager.cc:1918] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:18.766743 9986 log.cc:1196] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/6a6da0ba2d78499f9344b4baacbf6b5d
I20260807 15:02:18.766741 9415 catalog_manager.cc:4841] TS 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599): tablet 6a6da0ba2d78499f9344b4baacbf6b5d (table test-table [id=57d280e9ea484d31881c4b5f875cc9b8]) successfully deleted
I20260807 15:02:18.766980 9986 ts_tablet_manager.cc:1939] T 6a6da0ba2d78499f9344b4baacbf6b5d P 93f9cb6654d54820b23c1a344541ecad: Deleting consensus metadata
I20260807 15:02:18.767100 9551 tablet_service.cc:1467] Processing CreateTablet for tablet 23810550af1d46998b95d039b7ed9f0c (DEFAULT_TABLE table=test-table [id=5d0e4c01f703439e855f6ffef8444219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.767140 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 23810550af1d46998b95d039b7ed9f0c (DEFAULT_TABLE table=test-table [id=5d0e4c01f703439e855f6ffef8444219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.767223 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23810550af1d46998b95d039b7ed9f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.767230 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23810550af1d46998b95d039b7ed9f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.768555 9961 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Bootstrap starting.
I20260807 15:02:18.768639 9415 catalog_manager.cc:4841] TS 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053): tablet 6a6da0ba2d78499f9344b4baacbf6b5d (table test-table [id=57d280e9ea484d31881c4b5f875cc9b8]) successfully deleted
I20260807 15:02:18.768859 9415 catalog_manager.cc:4841] TS 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433): tablet 6a6da0ba2d78499f9344b4baacbf6b5d (table test-table [id=57d280e9ea484d31881c4b5f875cc9b8]) successfully deleted
I20260807 15:02:18.768923 9961 tablet_bootstrap.cc:654] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.768947 9717 tablet_service.cc:1467] Processing CreateTablet for tablet 23810550af1d46998b95d039b7ed9f0c (DEFAULT_TABLE table=test-table [id=5d0e4c01f703439e855f6ffef8444219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:18.769048 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23810550af1d46998b95d039b7ed9f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:18.769248 9966 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: Bootstrap starting.
I20260807 15:02:18.769640 9966 tablet_bootstrap.cc:654] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.770151 9961 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: No bootstrap required, opened a new log
I20260807 15:02:18.770161 9963 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Bootstrap starting.
I20260807 15:02:18.770208 9961 ts_tablet_manager.cc:1397] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:02:18.770332 9961 raft_consensus.cc:348] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.770465 9961 raft_consensus.cc:374] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.770500 9961 raft_consensus.cc:729] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f9cb6654d54820b23c1a344541ecad, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.770515 9963 tablet_bootstrap.cc:654] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:18.770555 9961 consensus_queue.cc:260] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.770669 9961 ts_tablet_manager.cc:1428] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.770932 9966 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: No bootstrap required, opened a new log
I20260807 15:02:18.771003 9966 ts_tablet_manager.cc:1397] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:02:18.771164 9963 tablet_bootstrap.cc:492] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: No bootstrap required, opened a new log
I20260807 15:02:18.771147 9966 raft_consensus.cc:348] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.771212 9963 ts_tablet_manager.cc:1397] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:18.771229 9966 raft_consensus.cc:374] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.771270 9966 raft_consensus.cc:729] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5ba5164a8c4333930af110c15db113, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.771317 9966 consensus_queue.cc:260] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.771317 9963 raft_consensus.cc:348] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.771391 9963 raft_consensus.cc:374] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:18.771412 9963 raft_consensus.cc:729] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Initialized, Role: FOLLOWER
I20260807 15:02:18.771422 9966 ts_tablet_manager.cc:1428] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:02:18.771457 9963 consensus_queue.cc:260] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.771555 9963 ts_tablet_manager.cc:1428] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:18.879160 9967 raft_consensus.cc:482] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:18.879293 9967 raft_consensus.cc:504] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.879468 9967 leader_election.cc:288] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.879987 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23810550af1d46998b95d039b7ed9f0c" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113" is_pre_election: true
I20260807 15:02:18.880016 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23810550af1d46998b95d039b7ed9f0c" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad" is_pre_election: true
I20260807 15:02:18.880131 9591 raft_consensus.cc:2454] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.880131 9886 raft_consensus.cc:2454] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 0.
I20260807 15:02:18.880347 9667 leader_election.cc:302] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 93f9cb6654d54820b23c1a344541ecad; no voters:
I20260807 15:02:18.880522 9967 raft_consensus.cc:2781] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:18.880596 9967 raft_consensus.cc:482] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:18.880632 9967 raft_consensus.cc:3037] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.881475 9967 raft_consensus.cc:504] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.881667 9967 leader_election.cc:288] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599), 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053)
I20260807 15:02:18.882004 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23810550af1d46998b95d039b7ed9f0c" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5ba5164a8c4333930af110c15db113"
I20260807 15:02:18.882040 9886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23810550af1d46998b95d039b7ed9f0c" candidate_uuid: "1fb26c0d50784fba94feedf7bda29ac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f9cb6654d54820b23c1a344541ecad"
I20260807 15:02:18.882118 9591 raft_consensus.cc:3037] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.882118 9886 raft_consensus.cc:3037] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:18.883102 9886 raft_consensus.cc:2454] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.883102 9591 raft_consensus.cc:2454] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26c0d50784fba94feedf7bda29ac9 in term 1.
I20260807 15:02:18.883373 9667 leader_election.cc:302] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26c0d50784fba94feedf7bda29ac9, 8b5ba5164a8c4333930af110c15db113; no voters:
I20260807 15:02:18.883510 9967 raft_consensus.cc:2781] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:18.883585 9967 raft_consensus.cc:686] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb26c0d50784fba94feedf7bda29ac9, State: Running, Role: LEADER
I20260807 15:02:18.883654 9967 consensus_queue.cc:237] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } }
I20260807 15:02:18.884416 9428 catalog_manager.cc:5495] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "1fb26c0d50784fba94feedf7bda29ac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26c0d50784fba94feedf7bda29ac9" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 } health_report { overall_health: UNKNOWN } } }
I20260807 15:02:18.894800 9886 raft_consensus.cc:1261] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.894866 9591 raft_consensus.cc:1261] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Refusing update from remote peer 1fb26c0d50784fba94feedf7bda29ac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:18.895164 9972 consensus_queue.cc:1035] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f9cb6654d54820b23c1a344541ecad" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.895339 9967 consensus_queue.cc:1035] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5ba5164a8c4333930af110c15db113" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:18.906550 9429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.906728 9429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:18.907696 9429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 506957b94b714f97bd01790155e8b0cd: Sending DeleteTablet for 3 replicas of tablet 23810550af1d46998b95d039b7ed9f0c
I20260807 15:02:18.908157 9717 tablet_service.cc:1514] Processing DeleteTablet for tablet 23810550af1d46998b95d039b7ed9f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57506
I20260807 15:02:18.908164 9551 tablet_service.cc:1514] Processing DeleteTablet for tablet 23810550af1d46998b95d039b7ed9f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:50346
I20260807 15:02:18.908190 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet 23810550af1d46998b95d039b7ed9f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:18 UTC) from {username='oryx', principal='oryx/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57210
I20260807 15:02:18.908283 9987 tablet_replica.cc:330] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: stopping tablet replica
I20260807 15:02:18.908288 9986 tablet_replica.cc:330] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: stopping tablet replica
I20260807 15:02:18.908337 9987 raft_consensus.cc:2229] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:18.908342 9986 raft_consensus.cc:2229] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.908371 9988 tablet_replica.cc:330] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113: stopping tablet replica
I20260807 15:02:18.908407 9987 raft_consensus.cc:2258] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.908417 9986 raft_consensus.cc:2258] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.908428 9988 raft_consensus.cc:2229] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:18.908469 9988 raft_consensus.cc:2258] T 23810550af1d46998b95d039b7ed9f0c P 8b5ba5164a8c4333930af110c15db113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:18.908936 9987 ts_tablet_manager.cc:1905] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.909099 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 9502
I20260807 15:02:18.909160 9986 ts_tablet_manager.cc:1905] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:18.911504 9986 ts_tablet_manager.cc:1918] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:02:18.911648 9986 log.cc:1196] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/23810550af1d46998b95d039b7ed9f0c
I20260807 15:02:18.911710 9987 ts_tablet_manager.cc:1918] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:02:18.911772 9987 log.cc:1196] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/23810550af1d46998b95d039b7ed9f0c
I20260807 15:02:18.911948 9986 ts_tablet_manager.cc:1939] T 23810550af1d46998b95d039b7ed9f0c P 93f9cb6654d54820b23c1a344541ecad: Deleting consensus metadata
I20260807 15:02:18.911975 9987 ts_tablet_manager.cc:1939] T 23810550af1d46998b95d039b7ed9f0c P 1fb26c0d50784fba94feedf7bda29ac9: Deleting consensus metadata
I20260807 15:02:18.912832 9415 catalog_manager.cc:4841] TS 93f9cb6654d54820b23c1a344541ecad (127.1.127.195:44053): tablet 23810550af1d46998b95d039b7ed9f0c (table test-table [id=5d0e4c01f703439e855f6ffef8444219]) successfully deleted
I20260807 15:02:18.913040 9415 catalog_manager.cc:4841] TS 1fb26c0d50784fba94feedf7bda29ac9 (127.1.127.194:41433): tablet 23810550af1d46998b95d039b7ed9f0c (table test-table [id=5d0e4c01f703439e855f6ffef8444219]) successfully deleted
W20260807 15:02:18.916904 9415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:34599 (error 108)
I20260807 15:02:18.917711 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:18.917379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34599 (local address 127.0.0.1:50354)
0807 15:02:18.917592 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:18.917658 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":59,"threads_started":1}
I20260807 15:02:18.917694 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 9649
W20260807 15:02:18.918722 9415 catalog_manager.cc:4584] TS 8b5ba5164a8c4333930af110c15db113 (127.1.127.193:34599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23810550af1d46998b95d039b7ed9f0c: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34599: connect: Connection refused (error 111)
I20260807 15:02:18.927116 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 9795
I20260807 15:02:18.935526 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 9397
2026-08-07T15:02:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2863 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:19 dist-test-slave-8tdl krb5kdc[10051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:19 dist-test-slave-8tdl krb5kdc[10051](info): set up 2 sockets
Aug 07 15:02:19 dist-test-slave-8tdl krb5kdc[10051](info): commencing operation
krb5kdc: starting...
W20260807 15:02:20.032562 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:20 dist-test-slave-8tdl krb5kdc[10051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:20.175311 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:44523
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:20.264989 10064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:20.265239 10064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:20.265265 10064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:20.266579 10064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:20.266629 10064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:20.266644 10064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:20.266656 10064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:20.266669 10064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:20.268383 10064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:20.268692 10064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:20.268949 10064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:20.274585 10064 server_base.cc:1034] running on GCE node
W20260807 15:02:20.274580 10072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:20.274655 10069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:20.274993 10070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:20.275383 10064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:20.275725 10064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:20.276891 10064 hybrid_clock.cc:648] HybridClock initialized: now 1786114940276917 us; error 75 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T15:02:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1345 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:20 dist-test-slave-8tdl krb5kdc[10077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:20 dist-test-slave-8tdl krb5kdc[10077](info): set up 2 sockets
Aug 07 15:02:20 dist-test-slave-8tdl krb5kdc[10077](info): commencing operation
krb5kdc: starting...
W20260807 15:02:21.399667 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:21 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:21.546592 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41405
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:21.636686 10090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:21.636968 10090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:21.637012 10090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:21.638298 10090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:21.638373 10090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:21.638396 10090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:21.638417 10090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:21.638448 10090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:21.640049 10090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:21.640360 10090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:21.640635 10090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:21.646111 10096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:21.646132 10098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:21.646130 10095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:21.646590 10090 server_base.cc:1034] running on GCE node
I20260807 15:02:21.646886 10090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:21.647241 10090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:21.648394 10090 hybrid_clock.cc:648] HybridClock initialized: now 1786114941648375 us; error 46 us; skew 500 ppm
Aug 07 15:02:21 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114941, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:21.659235 10090 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:21.659848 10090 webserver.cc:460] Webserver started at http://127.1.127.254:45163/ using document root <none> and password file <none>
I20260807 15:02:21.660138 10090 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:21.660218 10090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:21.660388 10090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:21.661623 10090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "15f39e42ba4f4f7194c88c70593392d3"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "27ab5a4cd3180ec1ec43c155f753bbb5"
server_key_iv: "60f018986b2e7cc0fae1f229e9241bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.662055 10090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "15f39e42ba4f4f7194c88c70593392d3"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "27ab5a4cd3180ec1ec43c155f753bbb5"
server_key_iv: "60f018986b2e7cc0fae1f229e9241bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.663873 10090 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:02:21.665098 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:21.665321 10090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:21.665437 10090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "15f39e42ba4f4f7194c88c70593392d3"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "27ab5a4cd3180ec1ec43c155f753bbb5"
server_key_iv: "60f018986b2e7cc0fae1f229e9241bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.665517 10090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:21.694301 10090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:21.697036 10090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:21.697177 10090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:21.701804 10090 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41405
I20260807 15:02:21.701894 10159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41405 every 8 connection(s)
I20260807 15:02:21.702286 10090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:21.703338 10160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:21.703632 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10090
I20260807 15:02:21.703790 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:21.704032 1535 external_mini_cluster.cc:1428] Setting key 0d817066f93224ebc669eb7fdd79919f
I20260807 15:02:21.706579 10160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Bootstrap starting.
I20260807 15:02:21.707314 10160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:21.707686 10160 log.cc:824] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:21.708637 10160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: No bootstrap required, opened a new log
Aug 07 15:02:21 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:21.711989 10160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:21.712375 10160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:21.712484 10160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f39e42ba4f4f7194c88c70593392d3, State: Initialized, Role: FOLLOWER
I20260807 15:02:21.712659 10160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:21.712754 10160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:21.712788 10160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:21.712846 10160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:21.713617 10160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:21.713722 10160 leader_election.cc:302] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f39e42ba4f4f7194c88c70593392d3; no voters:
I20260807 15:02:21.713971 10160 leader_election.cc:288] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:21.714010 10165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:21.714128 10165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 LEADER]: Becoming Leader. State: Replica: 15f39e42ba4f4f7194c88c70593392d3, State: Running, Role: LEADER
I20260807 15:02:21.714221 10165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:21.714285 10160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:21.714596 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15f39e42ba4f4f7194c88c70593392d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } } }
I20260807 15:02:21.714711 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:21.714880 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f39e42ba4f4f7194c88c70593392d3. Latest consensus state: current_term: 1 leader_uuid: "15f39e42ba4f4f7194c88c70593392d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } } }
I20260807 15:02:21.714956 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:21.715140 10170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:21.715749 10170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:21.717555 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:21.704661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.1.127.254:41405)
0807 15:02:21.704871 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:21.704879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:21.704908 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:02:21.705386 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:21.705393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:21.705445 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:21.705565 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:21.706419 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.707474 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.708546 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.709045 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.709138 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:21.712658 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:21.712665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:21.712665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:21.712682 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:21.715335 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:21.715913 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:21.715917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:21.715917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:21.716063 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:21.716353 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:21.716354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:21.716355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:21.716745 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:21.716913 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:21.717106 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:02:21.717277 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260807 15:02:21.718129 10170 catalog_manager.cc:1347] Generated new cluster ID: 3258b9ee8373473aa04ec11c5bc4cc6e
I20260807 15:02:21.718240 10170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:21.727975 10170 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:21.728757 10170 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:21.738112 10170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Generated new TSK 0
I20260807 15:02:21.738372 10170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:21.771562 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:21.861399 10188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:21.861682 10188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:21.861711 10188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:21.863190 10188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:21.863252 10188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:21.863284 10188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:21.864952 10188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:21.865258 10188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:21.865573 10188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:21.871488 10193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:21.871652 10196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:21.871765 10194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:21.872210 10188 server_base.cc:1034] running on GCE node
I20260807 15:02:21.872565 10188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:21.872900 10188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:21.874155 10188 hybrid_clock.cc:648] HybridClock initialized: now 1786114941874135 us; error 40 us; skew 500 ppm
Aug 07 15:02:21 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114941, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:21.877306 10188 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:21.877866 10188 webserver.cc:460] Webserver started at http://127.1.127.193:35173/ using document root <none> and password file <none>
I20260807 15:02:21.878126 10188 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:21.878176 10188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:21.878283 10188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:21.879613 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "472dc8421c014dc191bbc81f9f360ea5"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "615d7f29bba7252e0b5799ea129aaa98"
server_key_iv: "8344949e501418efe601e6eae4d72b2e"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.879985 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "472dc8421c014dc191bbc81f9f360ea5"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "615d7f29bba7252e0b5799ea129aaa98"
server_key_iv: "8344949e501418efe601e6eae4d72b2e"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.881542 10188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:02:21.882576 10204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:21.882856 10188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:21.882961 10188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "472dc8421c014dc191bbc81f9f360ea5"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "615d7f29bba7252e0b5799ea129aaa98"
server_key_iv: "8344949e501418efe601e6eae4d72b2e"
server_key_version: "encryptionkey@0"
I20260807 15:02:21.883050 10188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:21.898856 10188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:21.901429 10188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:21.901574 10188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:21.901913 10188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:21.902278 10188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:21.902315 10188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:21.902339 10188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:21.902352 10188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:21.909332 10188 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:44247
I20260807 15:02:21.909420 10323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:44247 every 8 connection(s)
I20260807 15:02:21.909788 10188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:02:21 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114941, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:21.918558 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10188
I20260807 15:02:21.918737 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:21.919044 1535 external_mini_cluster.cc:1428] Setting key 4b775503918d0f04217db3c038b080b2
I20260807 15:02:21.921718 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:21.910865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37595 (local address 127.1.127.254:41405)
0807 15:02:21.910913 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:21.910915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:21.911212 (+ 297us) server_negotiation.cc:407] Connection header received
0807 15:02:21.911821 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:21.911823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:21.911837 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:21.911893 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:21.913006 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.913542 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.914765 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.915052 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.915164 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:21.917914 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:21.917916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:21.917916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:21.917924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:21.919948 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:21.920568 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:21.920570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:21.920571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:21.920642 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:21.920958 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:21.920959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:21.920959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:21.921098 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:21.921188 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:21.921499 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 15:02:21.921604 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:02:21.921926 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:21.910808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:37595)
0807 15:02:21.911073 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:21.911117 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:21.911696 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:21.911996 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:21.912000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:21.912647 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:21.912870 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:21.912875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.913681 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.913681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:21.914626 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:21.914629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:21.915187 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:21.915188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:21.915341 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:21.915935 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:21.915945 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:21.917748 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:21.920140 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:21.920155 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:21.920158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:21.920453 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:21.920724 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:21.920725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:21.920725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:21.920878 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:21.921170 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:21.921180 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:21.921351 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 15:02:21.921474 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:02:21.921677 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260807 15:02:21.922897 10324 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41405
I20260807 15:02:21.923030 10324 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:21.923257 10324 heartbeater.cc:511] Master 127.1.127.254:41405 requested a full tablet report, sending...
I20260807 15:02:21.923938 10123 ts_manager.cc:194] Registered new tserver with Master: 472dc8421c014dc191bbc81f9f360ea5 (127.1.127.193:44247)
I20260807 15:02:21.924678 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:37595
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:21.959587 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:22.057531 10329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:22.057842 10329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:22.057886 10329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:22.059372 10329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:22.059465 10329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:22.059521 10329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:22.061305 10329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:22.061640 10329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:22.061924 10329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:22.067958 10329 server_base.cc:1034] running on GCE node
W20260807 15:02:22.067915 10334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.067899 10335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.067899 10337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:22.068347 10329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:22.068733 10329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:22.069931 10329 hybrid_clock.cc:648] HybridClock initialized: now 1786114942069916 us; error 29 us; skew 500 ppm
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:22.072898 10329 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:22.073422 10329 webserver.cc:460] Webserver started at http://127.1.127.194:44067/ using document root <none> and password file <none>
I20260807 15:02:22.073676 10329 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:22.073750 10329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:22.073911 10329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:22.075204 10329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "53c582540b0e41458c19f5e669bc2a15"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "2d672e7eeb3822990cb117e5e8ba0ec0"
server_key_iv: "d06b95ee04c1f212650bf44c09a7f1a7"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.075628 10329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "53c582540b0e41458c19f5e669bc2a15"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "2d672e7eeb3822990cb117e5e8ba0ec0"
server_key_iv: "d06b95ee04c1f212650bf44c09a7f1a7"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.077231 10329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:22.078401 10344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.078707 10329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:22.078811 10329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "53c582540b0e41458c19f5e669bc2a15"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "2d672e7eeb3822990cb117e5e8ba0ec0"
server_key_iv: "d06b95ee04c1f212650bf44c09a7f1a7"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.078883 10329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:22.104815 10329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:22.107568 10329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:22.107757 10329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:22.107970 10329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:22.108336 10329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:22.108369 10329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.108393 10329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:22.108405 10329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.114656 10329 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:36669
I20260807 15:02:22.114761 10463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:36669 every 8 connection(s)
I20260807 15:02:22.115160 10329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:22.116945 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10329
I20260807 15:02:22.117075 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:22.117293 1535 external_mini_cluster.cc:1428] Setting key 074d0454c11208b3269b3dcfc29024ea
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:22.127197 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.116000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:47791 (local address 127.1.127.254:41405)
0807 15:02:22.116085 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.116087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.116246 (+ 159us) server_negotiation.cc:407] Connection header received
0807 15:02:22.116792 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.116793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.116808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.116907 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.118424 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.119313 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.120728 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.120972 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.121035 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.123677 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:22.123678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:22.123679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:22.123694 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:22.125474 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:22.126092 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:22.126094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:22.126094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:22.126161 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:22.126440 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:22.126441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:22.126442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:22.126571 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:22.126648 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:22.126997 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 15:02:22.127102 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:02:22.127396 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.115931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:47791)
0807 15:02:22.116110 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.116150 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.116646 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.117105 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.117115 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.117957 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.118296 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.118303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.119427 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.119428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.120632 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.120636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.121067 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.121069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.121228 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.121777 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.121787 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.123561 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:22.125600 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:22.125610 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:22.125613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:22.126003 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:22.126237 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:22.126238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:22.126238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:22.126370 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:22.126650 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:22.126658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:22.126853 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 15:02:22.126962 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:02:22.127139 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260807 15:02:22.128214 10464 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41405
I20260807 15:02:22.128316 10464 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:22.128530 10464 heartbeater.cc:511] Master 127.1.127.254:41405 requested a full tablet report, sending...
I20260807 15:02:22.129021 10123 ts_manager.cc:194] Registered new tserver with Master: 53c582540b0e41458c19f5e669bc2a15 (127.1.127.194:36669)
I20260807 15:02:22.129426 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:47791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:22.151952 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:22.255453 10469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:22.255722 10469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:22.255753 10469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:22.257258 10469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:22.257323 10469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:22.257357 10469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:22.259248 10469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:22.259573 10469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:22.259851 10469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:22.266016 10475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.266037 10477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.266110 10474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:22.266642 10469 server_base.cc:1034] running on GCE node
I20260807 15:02:22.266945 10469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:22.267241 10469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:22.268417 10469 hybrid_clock.cc:648] HybridClock initialized: now 1786114942268386 us; error 47 us; skew 500 ppm
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:22.271353 10469 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:22.271893 10469 webserver.cc:460] Webserver started at http://127.1.127.195:41953/ using document root <none> and password file <none>
I20260807 15:02:22.272217 10469 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:22.272292 10469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:22.272442 10469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:22.273597 10469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "3af563857dc243dcaccb655db441dc4f"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "25399ee252e995831aaa36f30b487cfe"
server_key_iv: "3a3cd177699450f8f2a0717ae51f5d92"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.273996 10469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "3af563857dc243dcaccb655db441dc4f"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "25399ee252e995831aaa36f30b487cfe"
server_key_iv: "3a3cd177699450f8f2a0717ae51f5d92"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.275611 10469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:22.276572 10484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.276804 10469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:22.276922 10469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "3af563857dc243dcaccb655db441dc4f"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "25399ee252e995831aaa36f30b487cfe"
server_key_iv: "3a3cd177699450f8f2a0717ae51f5d92"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.277000 10469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:22.289942 10469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:22.292694 10469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:22.292975 10469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:22.293255 10469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:22.293623 10469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:22.293680 10469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.293725 10469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:22.293751 10469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.300406 10469 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38511
I20260807 15:02:22.300453 10605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38511 every 8 connection(s)
I20260807 15:02:22.300828 10469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:22.309154 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10469
I20260807 15:02:22.309288 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:22.309568 1535 external_mini_cluster.cc:1428] Setting key 0f13b4c878c3bfa930801cd9216256d4
I20260807 15:02:22.312292 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.301688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:48919 (local address 127.1.127.254:41405)
0807 15:02:22.301766 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.301769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.302079 (+ 310us) server_negotiation.cc:407] Connection header received
0807 15:02:22.302620 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.302622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.302634 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.302772 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.303870 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.304427 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.305559 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.305769 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.305832 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.308466 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:22.308468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:22.308469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:22.308479 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:22.310619 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:22.311187 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:22.311188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:22.311189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:22.311260 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:22.311612 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:22.311612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:22.311613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:22.311764 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:22.311826 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:22.312120 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 15:02:22.312205 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:02:22.312458 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.301678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:48919)
0807 15:02:22.301935 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.301980 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.302514 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.302870 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.302874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.303539 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.303748 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.303753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.304586 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.304588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.305448 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.305451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.305853 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.305854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.306086 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.306669 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.306754 (+ 85us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.308337 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:22.310751 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:22.310762 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:22.310764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:22.311083 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:22.311357 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:22.311358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:22.311358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:22.311507 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:22.311830 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:22.311837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:22.311972 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 15:02:22.312078 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:02:22.312218 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":122,"threads_started":1}
I20260807 15:02:22.313087 10606 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41405
I20260807 15:02:22.313181 10606 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:22.313339 10606 heartbeater.cc:511] Master 127.1.127.254:41405 requested a full tablet report, sending...
I20260807 15:02:22.313745 10123 ts_manager.cc:194] Registered new tserver with Master: 3af563857dc243dcaccb655db441dc4f (127.1.127.195:38511)
I20260807 15:02:22.314143 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:48919
I20260807 15:02:22.322286 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
Entry for principal oryx/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab.
WARNING: no policy specified for oryx/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
Entry for principal oryx/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab.
WARNING: no policy specified for oryx/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
Entry for principal oryx/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab.
I20260807 15:02:22.426281 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10188
I20260807 15:02:22.432580 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10329
I20260807 15:02:22.439600 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10469
I20260807 15:02:22.446456 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10090
I20260807 15:02:22.453537 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41405
--webserver_interface=127.1.127.254
--webserver_port=45163
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:22.549615 10615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:22.549860 10615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:22.549887 10615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:22.551215 10615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:22.551265 10615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:22.551280 10615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:22.551293 10615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:22.551306 10615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:22.552809 10615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=45163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:22.553031 10615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:22.553211 10615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:22.558565 10623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.558589 10621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.558565 10620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:22.558930 10615 server_base.cc:1034] running on GCE node
I20260807 15:02:22.559252 10615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:22.559672 10615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:22.560866 10615 hybrid_clock.cc:648] HybridClock initialized: now 1786114942560831 us; error 55 us; skew 500 ppm
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:22.563829 10615 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:22.564378 10615 webserver.cc:460] Webserver started at http://127.1.127.254:45163/ using document root <none> and password file <none>
I20260807 15:02:22.564659 10615 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:22.564728 10615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:22.566365 10615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:22.567405 10630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.567680 10615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:22.567791 10615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "15f39e42ba4f4f7194c88c70593392d3"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "27ab5a4cd3180ec1ec43c155f753bbb5"
server_key_iv: "60f018986b2e7cc0fae1f229e9241bea"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.568199 10615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:22.589979 10615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:22.592849 10615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:22.593048 10615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:22.597625 10615 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41405
I20260807 15:02:22.597671 10686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41405 every 8 connection(s)
I20260807 15:02:22.598095 10615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:22.599869 10687 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:02:22.600574 10687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Bootstrap starting.
I20260807 15:02:22.600692 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10615
I20260807 15:02:22.601024 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:44247
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:41405
--webserver_port=35173
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:02:22.610113 10687 log.cc:824] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:22.612417 10687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:02:22.612798 10687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Bootstrap complete.
I20260807 15:02:22.615633 10687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:22.615895 10687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f39e42ba4f4f7194c88c70593392d3, State: Initialized, Role: FOLLOWER
I20260807 15:02:22.616091 10687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:22.616171 10687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:22.616233 10687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:22.616287 10687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:02:22.617441 10687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:22.617542 10687 leader_election.cc:302] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f39e42ba4f4f7194c88c70593392d3; no voters:
I20260807 15:02:22.617805 10687 leader_election.cc:288] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:02:22.617856 10691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:02:22.618052 10691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [term 2 LEADER]: Becoming Leader. State: Replica: 15f39e42ba4f4f7194c88c70593392d3, State: Running, Role: LEADER
I20260807 15:02:22.618145 10687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:22.618227 10691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } }
I20260807 15:02:22.618630 10693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f39e42ba4f4f7194c88c70593392d3. Latest consensus state: current_term: 2 leader_uuid: "15f39e42ba4f4f7194c88c70593392d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } } }
I20260807 15:02:22.618644 10692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15f39e42ba4f4f7194c88c70593392d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f39e42ba4f4f7194c88c70593392d3" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41405 } } }
I20260807 15:02:22.618798 10693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:22.618808 10692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:22.619621 10700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:22.620213 10700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:22.620373 10700 catalog_manager.cc:1259] Loaded cluster ID: 3258b9ee8373473aa04ec11c5bc4cc6e
I20260807 15:02:22.620424 10700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:22.621436 10700 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:22.621560 10700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15f39e42ba4f4f7194c88c70593392d3: Loaded TSK: 0
I20260807 15:02:22.621698 10700 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:02:22.705081 10689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:22.705338 10689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:22.705382 10689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:22.706792 10689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:22.706872 10689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:22.706923 10689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:22.708619 10689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.193.keytab
--principal=oryx/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:44247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=35173
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:22.708922 10689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:22.709192 10689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:22.714772 10717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.714776 10714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.714774 10715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:22.715389 10689 server_base.cc:1034] running on GCE node
I20260807 15:02:22.715665 10689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:22.715971 10689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:22.717175 10689 hybrid_clock.cc:648] HybridClock initialized: now 1786114942717144 us; error 47 us; skew 500 ppm
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:22.720788 10689 init.cc:377] Logged in from keytab as oryx/127.1.127.193@KRBTEST.COM (short username oryx)
I20260807 15:02:22.721495 10689 webserver.cc:460] Webserver started at http://127.1.127.193:35173/ using document root <none> and password file <none>
I20260807 15:02:22.721779 10689 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:22.721875 10689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:22.723613 10689 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:02:22.724536 10724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.724828 10689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:22.725049 10689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "472dc8421c014dc191bbc81f9f360ea5"
format_stamp: "Formatted at 2026-08-07 15:02:21 on dist-test-slave-8tdl"
server_key: "615d7f29bba7252e0b5799ea129aaa98"
server_key_iv: "8344949e501418efe601e6eae4d72b2e"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.725431 10689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:22.741317 10689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:22.743968 10689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:22.744164 10689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:22.744454 10689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:22.744834 10689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:22.744890 10689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.744933 10689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:22.744966 10689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.751972 10689 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:44247
I20260807 15:02:22.752017 10843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:44247 every 8 connection(s)
I20260807 15:02:22.752398 10689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:22.758257 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10689
I20260807 15:02:22.758638 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:36669
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:41405
--webserver_port=44067
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:22.762750 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.753204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:56121)
0807 15:02:22.753392 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.753445 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.753936 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.754615 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.754620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.755339 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.755546 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.755552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.757212 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.757213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.758047 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.758050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.758982 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.758983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.759142 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.759709 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.759718 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.762459 (+ 2741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
W20260807 15:02:22.762792 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.753290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:56121 (local address 127.1.127.254:41405)
0807 15:02:22.753508 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.753517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.753556 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:02:22.754159 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.754167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.754227 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.754388 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.755692 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.757085 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.758156 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.758745 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.758870 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.762597 (+ 3727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:56121: BlockingRecv error: recv got EOF from 127.1.127.193:56121 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
W20260807 15:02:22.763330 10844 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41405 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.127.254:41405: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:22.768573 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.763611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:57655)
0807 15:02:22.763742 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.763754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.763800 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.763970 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.763971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.764114 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.764166 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.764168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.764876 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.764877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.765397 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.765399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.765793 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.765793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.765873 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.766585 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.766590 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.768474 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111}
W20260807 15:02:22.768631 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.763750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:57655 (local address 127.1.127.254:41405)
0807 15:02:22.763789 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.763791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.763801 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:22.763868 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.763869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.763881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.763966 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.764246 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.764792 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.765511 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.765707 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.765793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.768562 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:57655: BlockingRecv error: recv got EOF from 127.1.127.193:57655 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:22.775421 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.769166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:42325)
0807 15:02:22.769279 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.769291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.769343 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.769476 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.769476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.769617 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.769650 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.769652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.770280 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.770281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.770908 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.770911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.771271 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.771272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.771343 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.771811 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.771816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.775329 (+ 3513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260807 15:02:22.775475 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.769218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:42325 (local address 127.1.127.254:41405)
0807 15:02:22.769281 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.769284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.769327 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:02:22.769405 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.769406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.769416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.769471 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.769725 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.770200 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.771011 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.771199 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.771273 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.775411 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:42325: BlockingRecv error: recv got EOF from 127.1.127.193:42325 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 15:02:22.775885 10844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:02:22.858994 10849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:22.859236 10849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:22.859265 10849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:22.860512 10849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:22.860558 10849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:22.860589 10849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:22.862162 10849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.194.keytab
--principal=oryx/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:36669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=44067
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:22.862401 10849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:22.862605 10849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:22.868355 10855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.868360 10856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:22.868516 10858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:22.868515 10849 server_base.cc:1034] running on GCE node
I20260807 15:02:22.868810 10849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:22.869143 10849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:22.870312 10849 hybrid_clock.cc:648] HybridClock initialized: now 1786114942870280 us; error 42 us; skew 500 ppm
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114942, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:22.873494 10849 init.cc:377] Logged in from keytab as oryx/127.1.127.194@KRBTEST.COM (short username oryx)
I20260807 15:02:22.873950 10849 webserver.cc:460] Webserver started at http://127.1.127.194:44067/ using document root <none> and password file <none>
I20260807 15:02:22.874174 10849 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:22.874233 10849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:22.875816 10849 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:22.876618 10865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.876870 10849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:22.876992 10849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "53c582540b0e41458c19f5e669bc2a15"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "2d672e7eeb3822990cb117e5e8ba0ec0"
server_key_iv: "d06b95ee04c1f212650bf44c09a7f1a7"
server_key_version: "encryptionkey@0"
I20260807 15:02:22.877386 10849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:22.888494 10849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:22.891378 10849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:22.891574 10849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:22.891855 10849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:22.892216 10849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:22.892273 10849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.892323 10849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:22.892354 10849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:22.899168 10849 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:36669
I20260807 15:02:22.899259 10987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:36669 every 8 connection(s)
I20260807 15:02:22.899645 10849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:22.906641 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10849
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:22.907229 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:38511
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:41405
--webserver_port=41953
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:02:22.908856 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.900771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:41771 (local address 127.1.127.254:41405)
0807 15:02:22.900861 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.900864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.901112 (+ 248us) server_negotiation.cc:407] Connection header received
0807 15:02:22.901660 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.901661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.901676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.901747 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.902903 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.903426 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.904706 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.904918 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.904990 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.908804 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:41771: BlockingRecv error: recv got EOF from 127.1.127.194:41771 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:02:22.908946 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.900718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:41771)
0807 15:02:22.900938 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.900988 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.901508 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.901859 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.901864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.902534 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.902794 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.902800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.903553 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.903554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.904545 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.904584 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.905086 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.905087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.905217 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.905896 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.905906 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.908683 (+ 2777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
W20260807 15:02:22.909252 10989 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41405 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.127.254:41405: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:22.915663 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.909525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:49837)
0807 15:02:22.909657 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.909668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.909721 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.909869 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.909870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.909996 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.910110 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.910112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.911049 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.911050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.911595 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.911597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.912173 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.912174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.912269 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.912993 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.912998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.915595 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104}
W20260807 15:02:22.915845 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.909599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:49837 (local address 127.1.127.254:41405)
0807 15:02:22.909676 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.909679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.909710 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:02:22.909777 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.909778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.909790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.909856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.910193 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.910937 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.911737 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.911955 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.912171 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.915767 (+ 3596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:49837: BlockingRecv error: recv got EOF from 127.1.127.194:49837 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:22 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:02:22.923168 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:22.916228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:59731 (local address 127.1.127.254:41405)
0807 15:02:22.916289 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:22.916291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:22.916303 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:22.916366 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:22.916367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:22.916377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:22.916430 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:22.916898 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.917473 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.918790 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.919145 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.919213 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.923110 (+ 3897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:59731: BlockingRecv error: recv got EOF from 127.1.127.194:59731 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:02:22.923169 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:22.916148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:59731)
0807 15:02:22.916217 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:22.916232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:22.916292 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:22.916458 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:22.916460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:22.916664 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:22.916722 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.916724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.917638 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.917640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.918586 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:22.918591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:22.919241 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:22.919242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:22.919319 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:22.919960 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:22.919967 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:22.923001 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:02:22.923377 10989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:02:23.007228 10992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:23.007584 10992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:23.007615 10992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:23.009455 10992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:23.009534 10992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:23.009570 10992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:23.011390 10992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/krb5kdc/oryx_127.1.127.195.keytab
--principal=oryx/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:38511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=41953
--tserver_master_addrs=127.1.127.254:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:23.011703 10992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:23.012002 10992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:23.017786 11000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:23.017863 10998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:23.017863 10997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:23.018771 10992 server_base.cc:1034] running on GCE node
I20260807 15:02:23.019100 10992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:23.019455 10992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:23.020717 10992 hybrid_clock.cc:648] HybridClock initialized: now 1786114943020654 us; error 108 us; skew 500 ppm
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114943, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:23.023976 10992 init.cc:377] Logged in from keytab as oryx/127.1.127.195@KRBTEST.COM (short username oryx)
I20260807 15:02:23.024523 10992 webserver.cc:460] Webserver started at http://127.1.127.195:41953/ using document root <none> and password file <none>
I20260807 15:02:23.024775 10992 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:23.024866 10992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:23.026346 10992 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:02:23.027051 11008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:23.027257 10992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:23.027375 10992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "3af563857dc243dcaccb655db441dc4f"
format_stamp: "Formatted at 2026-08-07 15:02:22 on dist-test-slave-8tdl"
server_key: "25399ee252e995831aaa36f30b487cfe"
server_key_iv: "3a3cd177699450f8f2a0717ae51f5d92"
server_key_version: "encryptionkey@0"
I20260807 15:02:23.027782 10992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:23.043844 10992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:23.046257 10992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:23.046444 10992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:23.046802 10992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:23.047150 10992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:23.047209 10992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:23.047243 10992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:23.047271 10992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:23.053493 10992 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38511
I20260807 15:02:23.053539 11128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38511 every 8 connection(s)
I20260807 15:02:23.053978 10992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:23.055233 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 10992
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:02:23.062511 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:23.054865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:50291 (local address 127.1.127.254:41405)
0807 15:02:23.054945 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.054947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.055193 (+ 246us) server_negotiation.cc:407] Connection header received
0807 15:02:23.055988 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.055990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.056003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.056063 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.057167 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.057729 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.058807 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.059001 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.059077 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.062438 (+ 3361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:50291: BlockingRecv error: recv got EOF from 127.1.127.195:50291 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:02:23.062633 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.054896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:50291)
0807 15:02:23.055067 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:23.055105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:23.055866 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:23.056140 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:23.056145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:23.056853 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:23.057066 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.057071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.057836 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.057837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.058658 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.058660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.059090 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.059091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.059225 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.059705 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:23.059714 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:23.062362 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
W20260807 15:02:23.062906 11129 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41405 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.127.254:41405: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
I20260807 15:02:23.063674 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.055867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.1.127.254:41405)
0807 15:02:23.055980 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.055982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.055991 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:23.056112 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.056113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.056131 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.056207 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.056869 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.057424 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.058214 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.058465 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.058546 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.060054 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:23.060059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:23.060059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:23.060071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:23.062000 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:23.062382 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:23.062386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:23.062387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:23.062491 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:23.062859 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:23.062860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:23.062860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:23.063194 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:23.063330 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:23.063445 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 15:02:23.063564 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:23.068019 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.063154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:37961)
0807 15:02:23.063266 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:23.063277 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:23.063331 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:23.063477 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:23.063478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:23.063629 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:23.063691 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.063693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.064617 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.064618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.065154 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.065156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.065566 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.065567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.065615 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.066090 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:23.066094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:23.067970 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260807 15:02:23.068177 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:23.063218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:37961 (local address 127.1.127.254:41405)
0807 15:02:23.063257 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.063259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.063314 (+ 55us) server_negotiation.cc:407] Connection header received
0807 15:02:23.063389 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.063390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.063399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.063474 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.063799 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.064542 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.065251 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.065487 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.065560 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.068095 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:37961: BlockingRecv error: recv got EOF from 127.1.127.195:37961 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:23.072817 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.068411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:54193)
0807 15:02:23.068466 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:23.068477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:23.068537 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:23.068661 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:23.068662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:23.068790 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:23.068819 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.068820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.069550 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.069551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.070052 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.070054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.070436 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.070437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.070489 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.070968 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:23.070973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:23.072728 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:02:23.072865 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:23.068495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:54193 (local address 127.1.127.254:41405)
0807 15:02:23.068557 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.068559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.068567 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:02:23.068591 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.068592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.068600 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.068657 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.068954 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.069464 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.070150 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.070354 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.070426 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.072799 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:54193: BlockingRecv error: recv got EOF from 127.1.127.195:54193 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 15:02:23.072983 11129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:23.791486 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.776316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:55693)
0807 15:02:23.776453 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:23.776464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:23.776517 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:23.776694 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:23.776695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:23.776866 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:23.776919 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.776920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.777676 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.777678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.778391 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.778394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.778919 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.778920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.778972 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.786091 (+ 7119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:23.786098 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:23.791359 (+ 5261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260807 15:02:23.791638 11135 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:23.776412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:55693 (local address 127.1.127.254:41405)
0807 15:02:23.776567 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.776569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.776577 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:02:23.776599 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.776599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.776609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.776673 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.777041 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.777568 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.778496 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.778791 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.778856 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.791539 (+ 12683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:55693: BlockingRecv error: recv got EOF from 127.1.127.193:55693 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:23 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:23.929024 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:23.923854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:38549)
0807 15:02:23.924006 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:23.924018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:23.924078 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:23.924249 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:23.924250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:23.924424 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:23.924476 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.924478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.925304 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.925305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.925889 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:23.925891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.926320 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.926322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:23.926374 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.926903 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:23.926909 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:23.928926 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
W20260807 15:02:23.929201 11135 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:23.924044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:38549 (local address 127.1.127.254:41405)
0807 15:02:23.924119 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:23.924121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:23.924130 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:23.924155 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:23.924156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:23.924185 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:23.924236 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:23.924566 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.925207 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.926011 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:23.926222 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:23.926293 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:23.929131 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:38549: BlockingRecv error: recv got EOF from 127.1.127.194:38549 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:24.079309 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:24.073540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:46207)
0807 15:02:24.073860 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:24.073871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:24.073945 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:24.074124 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:24.074125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:24.074303 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:24.074408 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.074410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.075230 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.075231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.075822 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.075824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.076251 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.076252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.076305 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.076946 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:24.076953 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:24.079237 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":266,"threads_started":1}
W20260807 15:02:24.079527 11135 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:24.073658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46207 (local address 127.1.127.254:41405)
0807 15:02:24.073739 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:24.073742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:24.073957 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:02:24.073991 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:24.073992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:24.074007 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:24.074082 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:24.074516 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.075123 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.075918 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.076155 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.076218 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.079452 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:46207: BlockingRecv error: recv got EOF from 127.1.127.195:46207 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:24.797562 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:24.792154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:34001)
0807 15:02:24.792345 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:24.792357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:24.792421 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:24.792624 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:24.792625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:24.792800 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:24.792852 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.792854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.793566 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.793567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.794120 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.794122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.794574 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.794575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.794628 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.795180 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:24.795185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:24.797449 (+ 2264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
W20260807 15:02:24.797636 11139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:24.792306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34001 (local address 127.1.127.254:41405)
0807 15:02:24.792436 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:24.792438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:24.792449 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:24.792486 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:24.792487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:24.792498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:24.792596 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:24.792936 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.793462 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.794220 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.794458 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.794532 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.797553 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:34001: BlockingRecv error: recv got EOF from 127.1.127.193:34001 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:24 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:24.935884 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:24.929581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:50531)
0807 15:02:24.929762 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:24.929774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:24.929845 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:24.930191 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:24.930192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:24.930371 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:24.930425 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.930426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.931293 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.931294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.932078 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:24.932081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.932499 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.932500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:24.932574 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.933124 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:24.933129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:24.935801 (+ 2672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
W20260807 15:02:24.936115 11139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:24.929673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:50531 (local address 127.1.127.254:41405)
0807 15:02:24.929794 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:24.929796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:24.929832 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:02:24.929967 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:24.929969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:24.929981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:24.930110 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:24.930515 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.931189 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.932172 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:24.932392 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:24.932459 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:24.935945 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:50531: BlockingRecv error: recv got EOF from 127.1.127.194:50531 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:25.086537 11141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:25.079993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:49427)
0807 15:02:25.080235 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:25.080254 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:25.080321 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:25.080687 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:25.080688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:25.080909 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:25.081027 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.081030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.082182 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.082182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.082777 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.082779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.083278 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.083279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.083360 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.083943 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:25.083947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:25.086435 (+ 2488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":146,"threads_started":1}
W20260807 15:02:25.086603 11139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:25.080189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:49427 (local address 127.1.127.254:41405)
0807 15:02:25.080301 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:25.080303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:25.080315 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:25.080444 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:25.080446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:25.080476 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:25.080594 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:25.081191 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.082078 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.082894 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.083151 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.083218 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.086538 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:49427: BlockingRecv error: recv got EOF from 127.1.127.195:49427 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:25.803917 11142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:25.798290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:35763)
0807 15:02:25.798467 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:25.798482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:25.798569 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:25.798798 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:25.798800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:25.799074 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:25.799130 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.799132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.799955 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.799956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.800592 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.800595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.801023 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.801024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.801093 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.801667 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:25.801673 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:25.803851 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
W20260807 15:02:25.804195 11143 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:25.798358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:35763 (local address 127.1.127.254:41405)
0807 15:02:25.798504 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:25.798506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:25.798546 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:02:25.798658 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:25.798659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:25.798698 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:25.798771 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:25.799236 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.799848 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.800698 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.800907 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.800986 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.804135 (+ 3149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:35763: BlockingRecv error: recv got EOF from 127.1.127.193:35763 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:25 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:25.942871 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:25.936585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:38111)
0807 15:02:25.936740 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:25.936760 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:25.936822 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:25.937062 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:25.937064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:25.937243 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:25.937298 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.937300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.938106 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.938107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.938783 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:25.938787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.939393 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.939394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:25.939453 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.940154 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:25.940159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:25.942802 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
W20260807 15:02:25.943053 11143 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:25.936687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:38111 (local address 127.1.127.254:41405)
0807 15:02:25.936768 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:25.936770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:25.936811 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:02:25.936907 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:25.936908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:25.936920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:25.937006 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:25.937408 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.937926 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.938979 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:25.939200 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:25.939275 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:25.942950 (+ 3675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:38111: BlockingRecv error: recv got EOF from 127.1.127.194:38111 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:26.092614 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:26.087147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:58863)
0807 15:02:26.087342 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:26.087357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:26.087459 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:26.087674 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:26.087674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:26.087873 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:26.087924 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.087926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.088690 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.088691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.089291 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.089293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.089713 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.089714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.089786 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.090277 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:26.090281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:26.092553 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
W20260807 15:02:26.092715 11143 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:26.087247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:58863 (local address 127.1.127.254:41405)
0807 15:02:26.087342 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:26.087345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:26.087436 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:02:26.087565 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:26.087566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:26.087581 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:26.087665 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:26.088022 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.088587 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.089389 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.089611 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.089714 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.092644 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:58863: BlockingRecv error: recv got EOF from 127.1.127.195:58863 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:26.815488 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:26.804606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:58989)
0807 15:02:26.804760 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:26.804771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:26.804825 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:26.805100 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:26.805101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:26.805275 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:26.805328 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.805330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.806007 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.806008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.806656 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.806658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.807303 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.807303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.807363 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.813345 (+ 5982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:26.813358 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:26.815425 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260807 15:02:26.815707 11148 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:26.804687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:58989 (local address 127.1.127.254:41405)
0807 15:02:26.804869 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:26.804871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:26.804882 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:26.804919 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:26.804920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:26.804932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:26.805027 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:26.805409 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.805923 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.806800 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.806999 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.807063 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.815622 (+ 8559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:58989: BlockingRecv error: recv got EOF from 127.1.127.193:58989 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:26 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:26.949657 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:26.943554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:33809)
0807 15:02:26.943708 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:26.943719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:26.943773 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:26.943982 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:26.943983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:26.944158 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:26.944211 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.944213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.945059 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.945060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.945796 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:26.945799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.946604 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.946606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:26.946714 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.947253 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:26.947259 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:26.949570 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
W20260807 15:02:26.949764 11148 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:26.943660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:33809 (local address 127.1.127.254:41405)
0807 15:02:26.943759 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:26.943760 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:26.943772 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:26.943842 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:26.943843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:26.943856 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:26.943924 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:26.944282 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.944805 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.945899 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:26.946312 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:26.946528 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:26.949695 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:33809: BlockingRecv error: recv got EOF from 127.1.127.194:33809 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:27.098136 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:27.093167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:44019)
0807 15:02:27.093358 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:27.093370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:27.093438 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:27.093598 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:27.093599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:27.093780 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:27.093878 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.093880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.094600 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.094602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.095283 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.095287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.095670 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.095671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.095716 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.096261 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:27.096266 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:27.098069 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
W20260807 15:02:27.098335 11148 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:27.093224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:44019 (local address 127.1.127.254:41405)
0807 15:02:27.093285 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:27.093287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:27.093480 (+ 193us) server_negotiation.cc:407] Connection header received
0807 15:02:27.093508 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:27.093509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:27.093524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:27.093593 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:27.093981 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.094489 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.095378 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.095593 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.095653 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.098258 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:44019: BlockingRecv error: recv got EOF from 127.1.127.195:44019 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:27.821830 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:27.816235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:53587)
0807 15:02:27.816489 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:27.816507 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:27.816607 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:27.816803 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:27.816804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:27.816999 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:27.817060 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.817062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.817810 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.817811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.818490 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.818492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.818966 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.818967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.819042 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.819649 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:27.819655 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:27.821726 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
W20260807 15:02:27.821935 11152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:27.816383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:53587 (local address 127.1.127.254:41405)
0807 15:02:27.816498 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:27.816501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:27.816584 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:02:27.816680 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:27.816680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:27.816696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:27.816780 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:27.817151 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.817723 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.818599 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.818891 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.818965 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.821852 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:53587: BlockingRecv error: recv got EOF from 127.1.127.193:53587 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:27 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:27.956066 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:27.950555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:41203)
0807 15:02:27.950785 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:27.950798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:27.950906 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:27.951201 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:27.951203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:27.951401 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:27.951456 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.951458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.952188 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.952189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.952736 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:27.952738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.953138 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.953139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:27.953201 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.953713 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:27.953717 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:27.955998 (+ 2281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
W20260807 15:02:27.956239 11152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:27.950712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:41203 (local address 127.1.127.254:41405)
0807 15:02:27.950766 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:27.950769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:27.950883 (+ 114us) server_negotiation.cc:407] Connection header received
0807 15:02:27.950977 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:27.950978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:27.950995 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:27.951057 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:27.951565 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.952084 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.952829 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:27.953043 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:27.953126 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:27.956144 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:41203: BlockingRecv error: recv got EOF from 127.1.127.194:41203 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:28.103796 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:28.098721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:35521)
0807 15:02:28.098964 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:28.098975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:28.099050 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:28.099190 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:28.099190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:28.099365 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:28.099423 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.099425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.100156 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.100157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.100686 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.100688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.101141 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.101141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.101188 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.101694 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:28.101699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:28.103698 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":117,"threads_started":1}
W20260807 15:02:28.103878 11152 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:28.098784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:35521 (local address 127.1.127.254:41405)
0807 15:02:28.098840 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:28.098842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:28.099052 (+ 210us) server_negotiation.cc:407] Connection header received
0807 15:02:28.099099 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:28.099100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:28.099111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:28.099167 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:28.099489 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.100040 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.100768 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.101021 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.101081 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.103827 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:35521: BlockingRecv error: recv got EOF from 127.1.127.195:35521 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:28.828032 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:28.822471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:34757)
0807 15:02:28.822659 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:28.822670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:28.822747 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:28.823059 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:28.823061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:28.823261 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:28.823311 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.823314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.824106 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.824107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.824710 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.824712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.825151 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.825153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.825240 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.825858 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:28.825864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:28.827964 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":119,"threads_started":1}
W20260807 15:02:28.828231 11156 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:28.822548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34757 (local address 127.1.127.254:41405)
0807 15:02:28.822846 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:28.822848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:28.822858 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:28.822906 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:28.822906 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:28.822918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:28.822994 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:28.823426 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.824007 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.824795 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.825017 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.825092 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.828148 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:34757: BlockingRecv error: recv got EOF from 127.1.127.193:34757 (error 108)
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":225,"threads_started":1}
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:28 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:28.963054 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:28.956823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:46219)
0807 15:02:28.957006 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:28.957026 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:28.957082 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:28.957356 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:28.957357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:28.957555 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:28.957608 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.957609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.958576 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.958577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.959543 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:28.959547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.960003 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.960005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:28.960056 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.960679 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:28.960684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:28.962980 (+ 2296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":130,"threads_started":1}
W20260807 15:02:28.963200 11156 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:28.957030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:46219 (local address 127.1.127.254:41405)
0807 15:02:28.957186 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:28.957188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:28.957200 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:28.957237 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:28.957238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:28.957251 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:28.957323 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:28.957733 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.958382 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.959675 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:28.959894 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:28.959963 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:28.963118 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:46219: BlockingRecv error: recv got EOF from 127.1.127.194:46219 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:29.110441 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:29.104444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:53053)
0807 15:02:29.104581 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:29.104596 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:29.104742 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:29.104933 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:29.104934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:29.105126 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:29.105174 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.105176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.105952 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.105953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.106491 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.106492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.107037 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.107037 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.107084 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.107660 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:29.107667 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:29.110286 (+ 2619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
W20260807 15:02:29.110481 11156 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:29.104532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:53053 (local address 127.1.127.254:41405)
0807 15:02:29.104589 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:29.104591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:29.104717 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:02:29.104826 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:29.104827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:29.104842 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:29.104917 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:29.105265 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.105860 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.106587 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.106865 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.106945 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.110423 (+ 3478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:53053: BlockingRecv error: recv got EOF from 127.1.127.195:53053 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:29.834120 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:29.828660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:52563)
0807 15:02:29.828828 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:29.828840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:29.828910 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:29.829142 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:29.829143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:29.829328 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:29.829384 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.829386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.830095 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.830096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.830859 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.830863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.831248 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.831249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.831297 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.831990 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:29.832017 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:29.834043 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
W20260807 15:02:29.834240 11160 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:29.828755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:52563 (local address 127.1.127.254:41405)
0807 15:02:29.828993 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:29.828996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:29.829006 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:29.829040 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:29.829041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:29.829052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:29.829119 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:29.829466 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.830010 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.830942 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.831170 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.831235 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.834159 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:52563: BlockingRecv error: recv got EOF from 127.1.127.193:52563 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":152,"threads_started":1}
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:29 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:29.969429 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:29.963734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:37007)
0807 15:02:29.963897 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:29.963910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:29.964083 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:29.964248 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:29.964249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:29.964458 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:29.964509 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.964511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.965198 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.965200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.965778 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:29.965780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.966362 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.966363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:29.966434 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.967067 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:29.967072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:29.969328 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
W20260807 15:02:29.969523 11160 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:29.963794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:37007 (local address 127.1.127.254:41405)
0807 15:02:29.963864 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:29.963866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:29.964059 (+ 193us) server_negotiation.cc:407] Connection header received
0807 15:02:29.964147 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:29.964148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:29.964163 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:29.964239 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:29.964597 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.965109 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.965882 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:29.966235 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:29.966359 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:29.969455 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:37007: BlockingRecv error: recv got EOF from 127.1.127.194:37007 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:30.117470 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:30.111381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:55887)
0807 15:02:30.111662 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:30.111677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:30.111764 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:30.112093 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:30.112094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:30.112283 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:30.112336 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.112338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.113211 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.113213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.113910 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.113914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.114369 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.114370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.114429 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.115036 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:30.115041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:30.117353 (+ 2312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":212,"threads_started":1}
W20260807 15:02:30.117698 11160 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:30.111558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55887 (local address 127.1.127.254:41405)
0807 15:02:30.111668 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:30.111670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:30.111897 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:02:30.111943 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:30.111944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:30.111961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:30.112025 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:30.112426 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.113136 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.114031 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.114231 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.114315 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.117551 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:55887: BlockingRecv error: recv got EOF from 127.1.127.195:55887 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:30.840960 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:30.834831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:53923)
0807 15:02:30.834964 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:30.834977 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:30.835049 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:30.835320 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:30.835322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:30.835516 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:30.835568 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.835570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.836256 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.836257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.837370 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.837374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.837999 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.838001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.838075 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.838726 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:30.838733 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:30.840868 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260807 15:02:30.841046 11164 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:30.834978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:53923 (local address 127.1.127.254:41405)
0807 15:02:30.835150 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:30.835152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:30.835161 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:30.835187 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:30.835187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:30.835198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:30.835267 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:30.835655 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.836157 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.837500 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.837932 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.838017 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.840975 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:53923: BlockingRecv error: recv got EOF from 127.1.127.193:53923 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:30 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:30.976154 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:30.970136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:57559)
0807 15:02:30.970308 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:30.970319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:30.970383 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:30.970604 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:30.970606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:30.970851 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:30.970921 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.970979 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.971906 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.971907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.972497 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:30.972499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.973044 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.973045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:30.973109 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.973828 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:30.973833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:30.976030 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
W20260807 15:02:30.976209 11164 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:30.970246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57559 (local address 127.1.127.254:41405)
0807 15:02:30.970345 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:30.970346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:30.970361 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:02:30.970449 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:30.970450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:30.970461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:30.970568 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:30.971077 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.971812 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.972722 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:30.972916 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:30.972969 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:30.976159 (+ 3190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:57559: BlockingRecv error: recv got EOF from 127.1.127.194:57559 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:31.124511 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:31.118181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:55617)
0807 15:02:31.118360 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:31.118376 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:31.118439 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:31.118617 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:31.118617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:31.118900 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:31.118980 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.118984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.119711 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.119713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.120390 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.120393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.120916 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.120917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.120975 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.121602 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:31.121607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:31.124375 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
W20260807 15:02:31.124560 11164 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:31.118276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55617 (local address 127.1.127.254:41405)
0807 15:02:31.118339 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:31.118341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:31.118441 (+ 100us) server_negotiation.cc:407] Connection header received
0807 15:02:31.118517 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:31.118517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:31.118530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:31.118592 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:31.119057 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.119602 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.120544 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.120746 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.120821 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.124487 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:55617: BlockingRecv error: recv got EOF from 127.1.127.195:55617 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:31.852676 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:31.841645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:58551)
0807 15:02:31.841877 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:31.841890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:31.841956 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:31.842197 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:31.842198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:31.842384 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:31.842436 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.842438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.843376 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.843377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.843921 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.843923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.844417 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.844418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.844488 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.850094 (+ 5606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:31.850102 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:31.852610 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":117,"threads_started":1}
W20260807 15:02:31.852977 11168 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:31.841818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:58551 (local address 127.1.127.254:41405)
0807 15:02:31.842001 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:31.842003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:31.842014 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:31.842045 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:31.842045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:31.842064 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:31.842137 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:31.842574 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.843286 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.844090 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.844313 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.844384 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.852808 (+ 8424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:58551: BlockingRecv error: recv got EOF from 127.1.127.193:58551 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":74,"threads_started":1}
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:31 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:31.982183 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:31.976770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:37935)
0807 15:02:31.976948 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:31.976959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:31.977013 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:31.977160 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:31.977161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:31.977332 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:31.977382 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.977384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.978167 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.978169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.978771 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:31.978774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.979271 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.979272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:31.979324 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.979920 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:31.979925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:31.982116 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
W20260807 15:02:31.982456 11168 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:31.976909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:37935 (local address 127.1.127.254:41405)
0807 15:02:31.976983 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:31.976985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:31.977005 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:02:31.977080 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:31.977081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:31.977092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:31.977155 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:31.977481 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.978051 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.978957 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:31.979165 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:31.979232 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:31.982370 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:37935: BlockingRecv error: recv got EOF from 127.1.127.194:37935 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:32.131327 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:32.125228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:48781)
0807 15:02:32.125448 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:32.125462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:32.125538 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:32.125735 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:32.125735 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:32.125918 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:32.125970 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.125971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.126875 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.126876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.127470 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.127472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.127903 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.127904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.127959 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.128765 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:32.128770 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:32.131198 (+ 2428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
W20260807 15:02:32.131577 11168 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:32.125427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:48781 (local address 127.1.127.254:41405)
0807 15:02:32.125574 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:32.125576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:32.125588 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:32.125637 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:32.125638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:32.125651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:32.125709 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:32.126093 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.126615 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.127567 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.127779 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.127914 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.131314 (+ 3400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:48781: BlockingRecv error: recv got EOF from 127.1.127.195:48781 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:32.859670 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:32.853332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:54189)
0807 15:02:32.853529 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:32.853542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:32.853605 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:32.853797 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:32.853798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:32.854163 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:32.854242 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.854246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.855010 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.855011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.856103 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.856107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.856695 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.856695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.856750 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.857461 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:32.857467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:32.859604 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
W20260807 15:02:32.859947 11172 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:32.853450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:54189 (local address 127.1.127.254:41405)
0807 15:02:32.853586 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:32.853588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:32.853600 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:32.853670 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:32.853671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:32.853682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:32.853735 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:32.854345 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.854905 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.856209 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.856500 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.856615 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.859870 (+ 3255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:54189: BlockingRecv error: recv got EOF from 127.1.127.193:54189 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":113,"threads_started":1}
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:32 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:32.988130 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:32.982763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:50381)
0807 15:02:32.982917 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:32.982928 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:32.982987 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:32.983128 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:32.983129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:32.983301 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:32.983357 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.983359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.984128 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.984130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.984712 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:32.984715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.985175 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.985176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:32.985229 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.985820 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:32.985824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:32.988066 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260807 15:02:32.988286 11172 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:32.982841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:50381 (local address 127.1.127.254:41405)
0807 15:02:32.982919 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:32.982921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:32.982993 (+ 72us) server_negotiation.cc:407] Connection header received
0807 15:02:32.983041 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:32.983042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:32.983056 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:32.983121 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:32.983434 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.984023 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.984832 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:32.985081 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:32.985159 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:32.988197 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:50381: BlockingRecv error: recv got EOF from 127.1.127.194:50381 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:33.138253 11174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:33.132262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:44891)
0807 15:02:33.132478 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:33.132489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:33.132579 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:33.132743 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:33.132744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:33.132940 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:33.132989 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.132990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.133707 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.133708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.134518 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.134522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.135019 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.135020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.135085 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.135627 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:33.135632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:33.138158 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":107,"threads_started":1}
W20260807 15:02:33.138455 11172 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:33.132504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:44891 (local address 127.1.127.254:41405)
0807 15:02:33.132583 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:33.132585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:33.132613 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:02:33.132645 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:33.132646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:33.132667 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:33.132733 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:33.133097 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.133612 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.134640 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.134922 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.135018 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.138350 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:44891: BlockingRecv error: recv got EOF from 127.1.127.195:44891 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:33.867813 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:33.860274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:37945)
0807 15:02:33.860496 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:33.860509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:33.860591 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:33.860820 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:33.860821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:33.861093 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:33.861165 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.861168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.862223 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.862225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.863157 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.863161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.863647 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.863648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.863742 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.864583 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:33.864588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:33.867723 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":131,"threads_started":1}
W20260807 15:02:33.868028 11176 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:33.860338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37945 (local address 127.1.127.254:41405)
0807 15:02:33.860532 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:33.860535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:33.860570 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:02:33.860651 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:33.860652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:33.860668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:33.860764 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:33.861266 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.862100 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.863297 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.863554 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.863640 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.867863 (+ 4223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:37945: BlockingRecv error: recv got EOF from 127.1.127.193:37945 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:33 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:33.994210 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:33.988739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:43295)
0807 15:02:33.988927 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:33.988944 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:33.989027 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:33.989189 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:33.989189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:33.989383 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:33.989445 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.989447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.990192 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.990193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.990882 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:33.990884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.991266 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.991267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:33.991314 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.991890 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:33.991895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:33.994122 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
W20260807 15:02:33.994356 11176 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:33.988858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:43295 (local address 127.1.127.254:41405)
0807 15:02:33.988950 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:33.988951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:33.989020 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:02:33.989100 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:33.989101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:33.989113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:33.989161 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:33.989531 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.990092 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.990968 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:33.991183 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:33.991249 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:33.994258 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:43295: BlockingRecv error: recv got EOF from 127.1.127.194:43295 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:34.144791 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:34.138908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:55075)
0807 15:02:34.139099 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:34.139116 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:34.139210 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:34.139391 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:34.139392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:34.139618 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:34.139669 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.139672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.140780 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.140782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.141401 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.141404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.141818 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.141819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.141878 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:34.142476 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:34.142481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:34.144680 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
W20260807 15:02:34.144848 11176 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:34.139114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55075 (local address 127.1.127.254:41405)
0807 15:02:34.139200 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:34.139202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:34.139214 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:34.139286 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:34.139287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:34.139304 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:34.139381 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:34.139776 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.140646 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.141530 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.141737 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.141818 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:34.144778 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:55075: BlockingRecv error: recv got EOF from 127.1.127.195:55075 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:34.874776 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:34.868580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:34537)
0807 15:02:34.868794 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:34.868811 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:34.868877 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:34.869131 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:34.869132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:34.869315 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:34.869373 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.869376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.870146 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.870146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.870763 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.870766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.871260 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.871261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.871322 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:34.872280 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:34.872287 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:34.874624 (+ 2337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":120,"threads_started":1}
W20260807 15:02:34.874924 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:34.868709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34537 (local address 127.1.127.254:41405)
0807 15:02:34.868878 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:34.868880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:34.868911 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:02:34.868940 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:34.868941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:34.868953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:34.869007 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:34.869482 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.870047 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.870890 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.871137 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.871201 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:34.874802 (+ 3601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:34537: BlockingRecv error: recv got EOF from 127.1.127.193:34537 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:34 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:35.000175 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:34.994947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:51629)
0807 15:02:34.995188 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:34.995203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:34.995311 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:34.995505 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:34.995506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:34.995722 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:34.995770 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.995772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.996497 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.996499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.997042 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:34.997044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.997428 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.997430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:34.997485 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:34.998059 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:34.998065 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:35.000097 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":106,"threads_started":1}
W20260807 15:02:35.000434 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:34.995101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:51629 (local address 127.1.127.254:41405)
0807 15:02:34.995190 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:34.995193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:34.995288 (+ 95us) server_negotiation.cc:407] Connection header received
0807 15:02:34.995403 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:34.995404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:34.995424 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:34.995526 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:34.995863 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.996385 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.997149 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:34.997342 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:34.997435 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:35.000313 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:51629: BlockingRecv error: recv got EOF from 127.1.127.194:51629 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 15:02:35 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:35 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:35.150955 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:35.145384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:55345)
0807 15:02:35.145527 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:35.145541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:35.145606 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:35.145755 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:35.145756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:35.145931 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:35.146062 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:35.146064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.146865 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.146866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:35.147482 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:35.147485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.147968 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.147969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:35.148042 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:35.148659 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:35.148664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:35.150888 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":132,"threads_started":1}
W20260807 15:02:35.151203 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:35.145523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55345 (local address 127.1.127.254:41405)
0807 15:02:35.145600 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:35.145602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:35.145611 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:02:35.145675 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:35.145675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:35.145688 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:35.145734 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:35.146177 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.146749 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.147611 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.147811 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.147878 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:35.151127 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:55345: BlockingRecv error: recv got EOF from 127.1.127.195:55345 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 15:02:35 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:35 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:35.881659 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:35.875404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:45649)
0807 15:02:35.875620 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:35.875632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:35.875708 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:35.876033 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:35.876034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:35.876323 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:35.876395 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:35.876398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.877506 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.877508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:35.878062 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:35.878065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.878528 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.878529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:35.878580 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:35.879132 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:35.879137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:35.881566 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":134,"threads_started":1}
W20260807 15:02:35.881767 11184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:35.875518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:45649 (local address 127.1.127.254:41405)
0807 15:02:35.875649 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:35.875651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:35.875703 (+ 52us) server_negotiation.cc:407] Connection header received
0807 15:02:35.875767 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:35.875768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:35.875782 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:35.875971 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:35.876554 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.877370 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.878184 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:35.878406 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:35.878474 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:35.881688 (+ 3214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:45649: BlockingRecv error: recv got EOF from 127.1.127.193:45649 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:36.006749 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:36.000924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:37013)
0807 15:02:36.001067 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:36.001083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:36.001148 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:36.001467 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:36.001468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:36.001695 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:36.001746 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.001748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.002463 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.002464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.003090 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.003093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.003525 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.003526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.003581 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.004203 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:36.004217 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:36.006625 (+ 2408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
W20260807 15:02:36.006978 11184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:36.001227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:37013 (local address 127.1.127.254:41405)
0807 15:02:36.001280 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:36.001282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:36.001290 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:02:36.001328 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:36.001329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:36.001343 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:36.001420 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:36.001846 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.002361 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.003196 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.003425 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.003496 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.006907 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:37013: BlockingRecv error: recv got EOF from 127.1.127.194:37013 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:36.157011 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:36.151589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:46055)
0807 15:02:36.151771 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:36.151783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:36.151854 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:36.152001 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:36.152002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:36.152178 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:36.152229 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.152231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.152916 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.152917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.153531 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.153533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.153909 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.153910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.154007 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.154527 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:36.154531 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:36.156945 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
W20260807 15:02:36.157197 11184 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:36.151726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46055 (local address 127.1.127.254:41405)
0807 15:02:36.151788 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:36.151790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:36.151839 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:02:36.151912 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:36.151913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:36.151928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:36.151980 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:36.152303 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.152829 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.153627 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.153817 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.153885 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.157092 (+ 3207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:46055: BlockingRecv error: recv got EOF from 127.1.127.195:46055 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:36 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:36.892331 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:36.882333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:43169)
0807 15:02:36.882493 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:36.882508 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:36.882574 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:36.882782 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:36.882783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:36.882959 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:36.883024 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.883027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.883794 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.883796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.884497 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:36.884501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.884994 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.884995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:36.885052 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.889915 (+ 4863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:36.889922 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:36.892251 (+ 2329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
W20260807 15:02:36.892702 11188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:36.882422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:43169 (local address 127.1.127.254:41405)
0807 15:02:36.882612 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:36.882614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:36.882625 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:36.882654 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:36.882655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:36.882668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:36.882766 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:36.883131 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.883686 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.884627 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:36.884875 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:36.884952 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:36.892643 (+ 7691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:43169: BlockingRecv error: recv got EOF from 127.1.127.193:43169 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:37.013064 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:37.007439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:57533)
0807 15:02:37.007616 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:37.007632 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:37.007714 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:37.007914 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:37.007915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:37.008145 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:37.008211 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.008213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.008930 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.008932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.009508 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.009511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.009934 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.009935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.009996 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.010551 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:37.010556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:37.012980 (+ 2424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
W20260807 15:02:37.013190 11188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:37.007592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57533 (local address 127.1.127.254:41405)
0807 15:02:37.007681 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:37.007683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:37.007697 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:02:37.007812 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:37.007814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:37.007835 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:37.007904 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:37.008325 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.008835 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.009610 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.009820 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.009932 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.013108 (+ 3176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:57533: BlockingRecv error: recv got EOF from 127.1.127.194:57533 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.195@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:37.163512 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:37.157761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.195:46677)
0807 15:02:37.157976 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:37.157990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:37.158093 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:37.158266 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:37.158267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:37.158536 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:37.158599 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.158602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.159371 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.159372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.159960 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.159963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.160420 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.160421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.160474 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.160971 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:37.160977 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:37.163401 (+ 2424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":154,"threads_started":1}
W20260807 15:02:37.163590 11188 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:37.157870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46677 (local address 127.1.127.254:41405)
0807 15:02:37.157948 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:37.157950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:37.158108 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:02:37.158156 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:37.158156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:37.158175 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:37.158243 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:37.158695 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.159258 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.160073 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.160284 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.160351 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.163528 (+ 3177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.195:46677: BlockingRecv error: recv got EOF from 127.1.127.195:46677 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:37 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.193@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:37.898495 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:37.893047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.193:51625)
0807 15:02:37.893193 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:37.893206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:37.893280 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:37.893445 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:37.893446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:37.893662 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:37.893714 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.893717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.894537 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.894538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.895208 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:37.895212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.895622 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.895623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:37.895678 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.896332 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:37.896338 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:37.898429 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
W20260807 15:02:37.898682 11192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:37.893079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:51625 (local address 127.1.127.254:41405)
0807 15:02:37.893215 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:37.893218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:37.893280 (+ 62us) server_negotiation.cc:407] Connection header received
0807 15:02:37.893348 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:37.893348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:37.893361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:37.893427 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:37.893825 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.894439 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.895298 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:37.895521 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:37.895583 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:37.898584 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.193:51625: BlockingRecv error: recv got EOF from 127.1.127.193:51625 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Aug 07 15:02:38 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:02:38 dist-test-slave-8tdl krb5kdc[10077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.127.194@KRBTEST.COM for oryx/127.1.127.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:02:38.019552 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.013777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41405 (local address 127.1.127.194:36131)
0807 15:02:38.013974 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:38.013989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:38.014155 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:38.014327 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:38.014328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:38.014525 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:38.014577 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.014579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.015317 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.015319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.015855 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.015857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.016245 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.016246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.016322 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.017017 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:38.017056 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:38.019442 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.127.254:41405: Server oryx/127.1.127.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":120,"threads_started":1}
W20260807 15:02:38.019714 11192 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:38.013941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:36131 (local address 127.1.127.254:41405)
0807 15:02:38.013999 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.014002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.014130 (+ 128us) server_negotiation.cc:407] Connection header received
0807 15:02:38.014222 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.014223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.014239 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.014316 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.014669 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.015220 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.015950 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.016164 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.016247 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.019577 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.127.194:36131: BlockingRecv error: recv got EOF from 127.1.127.194:36131 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:02:38.056241 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10689
I20260807 15:02:38.063124 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10849
I20260807 15:02:38.069419 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10992
I20260807 15:02:38.075071 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 10615
2026-08-07T15:02:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17792 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T15:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:38Z Disabled control of system clock
I20260807 15:02:38.108144 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:38329
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:02:38.199221 11197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:38.199484 11197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:38.199510 11197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:38.200965 11197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:38.201028 11197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:38.201041 11197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:38.201054 11197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:38.201067 11197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:38.202705 11197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:38329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:38.203032 11197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:38.203394 11197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:38.208578 11203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.208570 11204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.208570 11206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.209134 11197 server_base.cc:1034] running on GCE node
I20260807 15:02:38.209434 11197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:38.209726 11197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:38.210902 11197 hybrid_clock.cc:648] HybridClock initialized: now 1786114958210886 us; error 46 us; skew 500 ppm
I20260807 15:02:38.212344 11197 webserver.cc:460] Webserver started at http://127.1.127.254:43355/ using document root <none> and password file <none>
I20260807 15:02:38.212641 11197 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:38.212716 11197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:38.212878 11197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:38.214038 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "4ba84b6cae35434baae9648abe894b67"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "175bef30aa1636f220b7821f6a001765"
server_key_iv: "9db190b5e131b268f7a53267d1bd3b06"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.214437 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "4ba84b6cae35434baae9648abe894b67"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "175bef30aa1636f220b7821f6a001765"
server_key_iv: "9db190b5e131b268f7a53267d1bd3b06"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.215991 11197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:38.216869 11212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.217068 11197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:38.217185 11197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "4ba84b6cae35434baae9648abe894b67"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "175bef30aa1636f220b7821f6a001765"
server_key_iv: "9db190b5e131b268f7a53267d1bd3b06"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.217267 11197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:38.239375 11197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:38.239789 11197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:38.239992 11197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:38.244634 11197 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:38329
I20260807 15:02:38.244684 11268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:38329 every 8 connection(s)
I20260807 15:02:38.245101 11197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:38.246264 11269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:38.248953 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67: Bootstrap starting.
I20260807 15:02:38.249706 11269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:38.250098 11269 log.cc:824] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:38.251125 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67: No bootstrap required, opened a new log
I20260807 15:02:38.252894 11269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } }
I20260807 15:02:38.253054 11269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:38.253078 11269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba84b6cae35434baae9648abe894b67, State: Initialized, Role: FOLLOWER
I20260807 15:02:38.253259 11269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } }
I20260807 15:02:38.253338 11269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:38.253364 11269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:38.253417 11269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:38.254103 11269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } }
I20260807 15:02:38.254237 11269 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba84b6cae35434baae9648abe894b67; no voters:
I20260807 15:02:38.254259 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11197
I20260807 15:02:38.254362 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:38.254446 11269 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:38.254515 11272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:38.254608 1535 external_mini_cluster.cc:1428] Setting key 3d71c51a803c1cd80a9da835402a3d4f
I20260807 15:02:38.254746 11272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [term 1 LEADER]: Becoming Leader. State: Replica: 4ba84b6cae35434baae9648abe894b67, State: Running, Role: LEADER
I20260807 15:02:38.254779 11269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:38.254864 11272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } }
I20260807 15:02:38.255373 11272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba84b6cae35434baae9648abe894b67. Latest consensus state: current_term: 1 leader_uuid: "4ba84b6cae35434baae9648abe894b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } } }
I20260807 15:02:38.255498 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba84b6cae35434baae9648abe894b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba84b6cae35434baae9648abe894b67" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 38329 } } }
I20260807 15:02:38.255648 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:38.255579 11272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:38.255972 11284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:38.256623 11284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:38.258363 11284 catalog_manager.cc:1347] Generated new cluster ID: 221ac5b829344573ba5a0a1fa0fbe365
I20260807 15:02:38.258414 11284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:38.262068 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.255555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.1.127.254:38329)
0807 15:02:38.256487 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.256497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.256532 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:02:38.256670 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.256678 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.256759 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.257275 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.257382 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.258576 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.259757 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.260409 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.260535 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.261320 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:38.261324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:38.261325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:38.261546 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:38.261590 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:38.261605 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:38.261667 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:38.261768 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:02:38.261849 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":95,"threads_started":1}
I20260807 15:02:38.267939 11284 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:38.268564 11284 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:38.282338 11284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ba84b6cae35434baae9648abe894b67: Generated new TSK 0
I20260807 15:02:38.282631 11284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:38.289158 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--builtin_ntp_servers=127.1.127.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:02:38.391554 11294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:38.391834 11294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:38.391885 11294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:38.393215 11294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:38.393292 11294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:38.393335 11294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:38.394989 11294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:38.395438 11294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:38.395860 11294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:38.402788 11300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.402788 11299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.405047 11302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.406683 11294 server_base.cc:1034] running on GCE node
I20260807 15:02:38.407006 11294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:38.407323 11294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:38.408511 11294 hybrid_clock.cc:648] HybridClock initialized: now 1786114958408472 us; error 57 us; skew 500 ppm
I20260807 15:02:38.410269 11294 webserver.cc:460] Webserver started at http://127.1.127.193:46275/ using document root <none> and password file <none>
I20260807 15:02:38.410563 11294 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:38.410614 11294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:38.410830 11294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:38.412050 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "66e5629c23524ecbbc915c56a5b140eb"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "fa18511628ef22ff09cb3149d891a7cf"
server_key_iv: "2927970f5d5c9a77f3833408d92b7e2c"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.412464 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "66e5629c23524ecbbc915c56a5b140eb"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "fa18511628ef22ff09cb3149d891a7cf"
server_key_iv: "2927970f5d5c9a77f3833408d92b7e2c"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.414002 11294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:38.414927 11308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.415185 11294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:38.415289 11294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "66e5629c23524ecbbc915c56a5b140eb"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "fa18511628ef22ff09cb3149d891a7cf"
server_key_iv: "2927970f5d5c9a77f3833408d92b7e2c"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.415356 11294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:38.424556 11294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:38.424866 11294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:38.424979 11294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:38.425196 11294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:38.425489 11294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:38.425520 11294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.425544 11294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:38.425556 11294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.431617 11294 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:43195
I20260807 15:02:38.431705 11431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:43195 every 8 connection(s)
I20260807 15:02:38.432083 11294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:38.438210 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.433063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:50031 (local address 127.1.127.254:38329)
0807 15:02:38.433136 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.433138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.433377 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:02:38.433967 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.433969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.433985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.434048 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.435007 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.435537 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.436583 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.436799 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.436925 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.437921 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:38.437922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:38.437923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:38.437930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:38.437948 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:38.437953 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:38.438009 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:38.438089 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:02:38.438128 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:02:38.438436 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.433043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:38329 (local address 127.1.127.193:50031)
0807 15:02:38.433264 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:38.433300 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:38.433806 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:38.434151 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:38.434155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:38.434164 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:38.434797 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.434803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.435635 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.435636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.436471 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.436474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.436942 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.436943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.437083 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.437694 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:38.437694 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:38.437726 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:38.437752 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:38.438020 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:38.438025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:38.438028 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:38.438082 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:02:38.438178 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260807 15:02:38.438642 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11294
I20260807 15:02:38.438808 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:38.439113 1535 external_mini_cluster.cc:1428] Setting key d0327b3c02c508d523e11b63f2bb8de5
I20260807 15:02:38.439231 11432 heartbeater.cc:347] Connected to a master server at 127.1.127.254:38329
I20260807 15:02:38.439369 11432 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:38.439599 11432 heartbeater.cc:511] Master 127.1.127.254:38329 requested a full tablet report, sending...
I20260807 15:02:38.440215 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--builtin_ntp_servers=127.1.127.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 15:02:38.440294 11231 ts_manager.cc:194] Registered new tserver with Master: 66e5629c23524ecbbc915c56a5b140eb (127.1.127.193:43195)
I20260807 15:02:38.441215 11231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.193:50031
W20260807 15:02:38.531095 11435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:38.531335 11435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:38.531361 11435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:38.532647 11435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:38.532696 11435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:38.532722 11435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:38.534138 11435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:38.534365 11435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:38.534654 11435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:38.540796 11441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.540732 11440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.540764 11435 server_base.cc:1034] running on GCE node
W20260807 15:02:38.540728 11444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.541278 11435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:38.541585 11435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:38.542783 11435 hybrid_clock.cc:648] HybridClock initialized: now 1786114958542751 us; error 32 us; skew 500 ppm
I20260807 15:02:38.544407 11435 webserver.cc:460] Webserver started at http://127.1.127.194:40849/ using document root <none> and password file <none>
I20260807 15:02:38.544682 11435 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:38.544757 11435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:38.544919 11435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:38.546108 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "420a1e62a7c84c04878644b4bb9422b8"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "0fd65c706074d9b3f76b1e7eb16ceb6b"
server_key_iv: "def92269b6e5659e86235a435ec0c7eb"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.546564 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "420a1e62a7c84c04878644b4bb9422b8"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "0fd65c706074d9b3f76b1e7eb16ceb6b"
server_key_iv: "def92269b6e5659e86235a435ec0c7eb"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.548170 11435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:38.549513 11450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.549785 11435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:38.549898 11435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "420a1e62a7c84c04878644b4bb9422b8"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "0fd65c706074d9b3f76b1e7eb16ceb6b"
server_key_iv: "def92269b6e5659e86235a435ec0c7eb"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.549991 11435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:38.558980 11435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:38.559314 11435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:38.559435 11435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:38.559690 11435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:38.560071 11435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:38.560114 11435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.560140 11435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:38.560153 11435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.566838 11435 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:38269
I20260807 15:02:38.566888 11569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:38269 every 8 connection(s)
I20260807 15:02:38.567301 11435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:38.572706 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.568214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:58865 (local address 127.1.127.254:38329)
0807 15:02:38.568291 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.568293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.568482 (+ 189us) server_negotiation.cc:407] Connection header received
0807 15:02:38.568952 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.568953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.568964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.569024 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.569688 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.570224 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.571354 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.571546 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.571616 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.572423 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:38.572424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:38.572425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:38.572429 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:38.572443 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:38.572447 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:38.572495 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:38.572611 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 15:02:38.572641 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:02:38.572872 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.568113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:38329 (local address 127.1.127.194:58865)
0807 15:02:38.568317 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:38.568359 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:38.568858 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:38.569088 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:38.569092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:38.569100 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:38.569607 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.569612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.570325 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.570326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.571260 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.571263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.571637 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.571638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.571760 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.572284 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:38.572284 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:38.572313 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:38.572336 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:38.572503 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:38.572507 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:38.572510 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:38.572564 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:02:38.572649 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260807 15:02:38.573625 11570 heartbeater.cc:347] Connected to a master server at 127.1.127.254:38329
I20260807 15:02:38.573742 11570 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:38.573961 11570 heartbeater.cc:511] Master 127.1.127.254:38329 requested a full tablet report, sending...
I20260807 15:02:38.574589 11231 ts_manager.cc:194] Registered new tserver with Master: 420a1e62a7c84c04878644b4bb9422b8 (127.1.127.194:38269)
I20260807 15:02:38.575026 11231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.194:58865
I20260807 15:02:38.576683 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11435
I20260807 15:02:38.576761 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:38.576977 1535 external_mini_cluster.cc:1428] Setting key 25fc765a4a5ef399dd4134549b46c141
I20260807 15:02:38.578069 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--builtin_ntp_servers=127.1.127.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:02:38.663337 11574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:38.663579 11574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:38.663604 11574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:38.664887 11574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:38.664945 11574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:38.664974 11574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:38.666525 11574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:38.666857 11574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:38.667196 11574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:02:38.673269 11574 server_base.cc:1034] running on GCE node
W20260807 15:02:38.673223 11580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.673249 11581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.673243 11583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.673729 11574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:38.674074 11574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:38.675277 11574 hybrid_clock.cc:648] HybridClock initialized: now 1786114958675258 us; error 31 us; skew 500 ppm
I20260807 15:02:38.676874 11574 webserver.cc:460] Webserver started at http://127.1.127.195:39727/ using document root <none> and password file <none>
I20260807 15:02:38.677261 11574 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:38.677314 11574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:38.677419 11574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:38.678589 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "67e4f446d6c24e47af4df1f1e684f5fd"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "d2ec9213f168fa46abc185195966a0a1"
server_key_iv: "813c73dc5d9afdb21229e976c72dd9d3"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.679025 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "67e4f446d6c24e47af4df1f1e684f5fd"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "d2ec9213f168fa46abc185195966a0a1"
server_key_iv: "813c73dc5d9afdb21229e976c72dd9d3"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.680496 11574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:38.681438 11589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.681730 11574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:38.681865 11574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "67e4f446d6c24e47af4df1f1e684f5fd"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "d2ec9213f168fa46abc185195966a0a1"
server_key_iv: "813c73dc5d9afdb21229e976c72dd9d3"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.681963 11574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:38.690536 11574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:38.690965 11574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:38.691154 11574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:38.691447 11574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:38.691874 11574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:38.691938 11574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.691978 11574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:38.692027 11574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.698999 11574 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:39519
I20260807 15:02:38.699060 11707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:39519 every 8 connection(s)
I20260807 15:02:38.699467 11574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:38.704399 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11574
I20260807 15:02:38.704634 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:38.704873 1535 external_mini_cluster.cc:1428] Setting key f8c6b839db42d06c81ebaf33734c8a8b
I20260807 15:02:38.705677 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.700298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:52639 (local address 127.1.127.254:38329)
0807 15:02:38.700385 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.700388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.700698 (+ 310us) server_negotiation.cc:407] Connection header received
0807 15:02:38.701264 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.701265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.701278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.701345 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.702154 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.702732 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.703806 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.704034 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.704105 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.705271 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:38.705273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:38.705273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:38.705278 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:38.705297 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:38.705303 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:38.705367 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:38.705542 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 15:02:38.705591 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:02:38.705893 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.700283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:38329 (local address 127.1.127.195:52639)
0807 15:02:38.700532 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:38.700576 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:38.701145 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:38.701421 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:38.701424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:38.701433 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:38.702038 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.702044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.702826 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.702827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.703698 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:38.703701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:38.704131 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:38.704132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:38.704268 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:38.705094 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:38.705095 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:38.705141 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:38.705172 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:38.705406 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:38.705414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:38.705417 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:38.705494 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:02:38.705614 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":132,"threads_started":1}
I20260807 15:02:38.706501 11708 heartbeater.cc:347] Connected to a master server at 127.1.127.254:38329
I20260807 15:02:38.706615 11708 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:38.706851 11708 heartbeater.cc:511] Master 127.1.127.254:38329 requested a full tablet report, sending...
I20260807 15:02:38.707294 11231 ts_manager.cc:194] Registered new tserver with Master: 67e4f446d6c24e47af4df1f1e684f5fd (127.1.127.195:39519)
I20260807 15:02:38.707684 11231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.195:52639
I20260807 15:02:38.717641 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:02:38.720054 11283 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:38.719352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42432 (local address 127.1.127.254:38329)
0807 15:02:38.719397 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.719399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.719637 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:02:38.719687 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.719687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.719699 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.719755 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.719939 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42432: BlockingRecv error: recv got EOF from 127.0.0.1:42432 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:02:38.720176 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11294
I20260807 15:02:38.728317 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11435
I20260807 15:02:38.735106 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11574
I20260807 15:02:38.740610 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11197
2026-08-07T15:02:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (664 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T15:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:38Z Disabled control of system clock
I20260807 15:02:38.773056 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:34113
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:33401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:02:38.859910 11721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:38.860163 11721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:38.860188 11721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:38.861455 11721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:38.861510 11721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:38.861523 11721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:38.861536 11721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:38.861548 11721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:38.863122 11721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:34113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:38.863365 11721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:38.863649 11721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:38.869333 11726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.869513 11721 server_base.cc:1034] running on GCE node
W20260807 15:02:38.869578 11729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:38.869402 11727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:38.870049 11721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:38.870471 11721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:38.871652 11721 hybrid_clock.cc:648] HybridClock initialized: now 1786114958871621 us; error 46 us; skew 500 ppm
I20260807 15:02:38.873169 11721 webserver.cc:460] Webserver started at http://127.1.127.254:46563/ using document root <none> and password file <none>
I20260807 15:02:38.873462 11721 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:38.873561 11721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:38.873739 11721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:38.875126 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "711807e562b34a7aaf4f4afdd975e521"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "ff23bf78ea7c57c63dd4c974e1e794be"
server_key_iv: "70533c4d57ce5f105cb5522a1ac0ca27"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.875531 11721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "711807e562b34a7aaf4f4afdd975e521"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "ff23bf78ea7c57c63dd4c974e1e794be"
server_key_iv: "70533c4d57ce5f105cb5522a1ac0ca27"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.877036 11721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:38.878158 11735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:38.878494 11721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:38.878628 11721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "711807e562b34a7aaf4f4afdd975e521"
format_stamp: "Formatted at 2026-08-07 15:02:38 on dist-test-slave-8tdl"
server_key: "ff23bf78ea7c57c63dd4c974e1e794be"
server_key_iv: "70533c4d57ce5f105cb5522a1ac0ca27"
server_key_version: "encryptionkey@0"
I20260807 15:02:38.878777 11721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:38.883077 11721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:38.883462 11721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:38.883638 11721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:38.889765 11721 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:34113
I20260807 15:02:38.889798 11790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:34113 every 8 connection(s)
I20260807 15:02:38.890326 11721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:38.891227 11791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:38.894310 11791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: Bootstrap starting.
I20260807 15:02:38.895087 11791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:38.895377 11791 log.cc:824] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:38.896190 11791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: No bootstrap required, opened a new log
I20260807 15:02:38.897972 11791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } }
I20260807 15:02:38.898149 11791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:38.898173 11791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711807e562b34a7aaf4f4afdd975e521, State: Initialized, Role: FOLLOWER
I20260807 15:02:38.898293 11791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } }
I20260807 15:02:38.898391 11791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:38.898432 11791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:38.898486 11791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:38.898901 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11721
I20260807 15:02:38.899001 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:38.899231 1535 external_mini_cluster.cc:1428] Setting key d5099552c0567dec17fee35ecbcdbe94
I20260807 15:02:38.899485 11791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } }
I20260807 15:02:38.899632 11791 leader_election.cc:302] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 711807e562b34a7aaf4f4afdd975e521; no voters:
I20260807 15:02:38.899844 11791 leader_election.cc:288] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:38.899955 11794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:38.900216 11794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [term 1 LEADER]: Becoming Leader. State: Replica: 711807e562b34a7aaf4f4afdd975e521, State: Running, Role: LEADER
I20260807 15:02:38.900230 11791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:38.900425 11794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } }
I20260807 15:02:38.901902 11798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 711807e562b34a7aaf4f4afdd975e521. Latest consensus state: current_term: 1 leader_uuid: "711807e562b34a7aaf4f4afdd975e521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } } }
I20260807 15:02:38.902096 11797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "711807e562b34a7aaf4f4afdd975e521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711807e562b34a7aaf4f4afdd975e521" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 34113 } } }
I20260807 15:02:38.902182 11798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [sys.catalog]: This master's current role is: LEADER
W20260807 15:02:38.902558 11811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:02:38.902602 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:38.899867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.1.127.254:34113)
0807 15:02:38.900134 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:38.900144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:38.900181 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:02:38.900330 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:38.900338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:38.900383 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:38.900866 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:38.901921 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:38.901925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:38.901926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:38.901929 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:38.901962 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:38.901976 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:38.902227 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:38.902263 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 15:02:38.902376 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
W20260807 15:02:38.902608 11811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:02:38.902427 11797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521 [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:38.902949 11814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:38.903194 11814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:38.905093 11814 catalog_manager.cc:1347] Generated new cluster ID: 237b53e7c9ab4a39ba1643df30e0ce55
I20260807 15:02:38.905171 11814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:38.913206 11814 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:38.913466 11814 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:38.917198 11814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 711807e562b34a7aaf4f4afdd975e521: Generated new TSK 0
I20260807 15:02:38.917474 11814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:02:38.920671 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--builtin_ntp_servers=127.1.127.212:33401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:02:39.014801 11816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:39.015090 11816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:39.015132 11816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:39.016522 11816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:39.016613 11816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:39.016659 11816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:39.018574 11816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:39.018980 11816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:39.019488 11816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:39.026007 11822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:39.026028 11824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:39.025982 11821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:39.026584 11816 server_base.cc:1034] running on GCE node
I20260807 15:02:39.026863 11816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:39.027158 11816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:39.028340 11816 hybrid_clock.cc:648] HybridClock initialized: now 1786114959028314 us; error 41 us; skew 500 ppm
I20260807 15:02:39.029886 11816 webserver.cc:460] Webserver started at http://127.1.127.193:34145/ using document root <none> and password file <none>
I20260807 15:02:39.030256 11816 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:39.030332 11816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:39.030504 11816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:39.031778 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "c99a2a0ceef44efea055e87090e227b9"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f0bfe40f241876683b36285622a6c74"
server_key_iv: "0651b065e7f8aae94748f3c143973f6b"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.032390 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "c99a2a0ceef44efea055e87090e227b9"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f0bfe40f241876683b36285622a6c74"
server_key_iv: "0651b065e7f8aae94748f3c143973f6b"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.034016 11816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:39.034966 11830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.035164 11816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:39.035283 11816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "c99a2a0ceef44efea055e87090e227b9"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f0bfe40f241876683b36285622a6c74"
server_key_iv: "0651b065e7f8aae94748f3c143973f6b"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.035367 11816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:39.038919 11816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:39.039224 11816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:39.039376 11816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:39.039595 11816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:39.039968 11816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:39.040040 11816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.040086 11816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:39.040113 11816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.046954 11816 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35151
I20260807 15:02:39.046998 11949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35151 every 8 connection(s)
I20260807 15:02:39.047379 11816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:02:39.050223 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11816
I20260807 15:02:39.050349 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:39.050565 1535 external_mini_cluster.cc:1428] Setting key a521d46ad86bad4ca99948af4800465e
I20260807 15:02:39.051005 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.048192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:52183 (local address 127.1.127.254:34113)
0807 15:02:39.048296 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:39.048299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:39.048530 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:02:39.049169 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:39.049171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:39.049178 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:39.049237 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:39.050563 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:39.050565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:39.050565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:39.050571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:39.050592 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:39.050596 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:39.050714 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:39.050897 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 15:02:39.050920 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:02:39.051168 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.048156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34113 (local address 127.1.127.193:52183)
0807 15:02:39.048391 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:39.048428 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:39.049027 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:39.049311 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:39.049315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:39.049325 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:39.050312 (+ 987us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:39.050312 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:39.050384 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:39.050426 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:39.050786 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:39.050800 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:39.050804 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:39.050884 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:02:39.050944 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260807 15:02:39.051870 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--builtin_ntp_servers=127.1.127.212:33401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 15:02:39.051911 11950 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34113
I20260807 15:02:39.052045 11950 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:39.052210 11950 heartbeater.cc:511] Master 127.1.127.254:34113 requested a full tablet report, sending...
I20260807 15:02:39.052886 11753 ts_manager.cc:194] Registered new tserver with Master: c99a2a0ceef44efea055e87090e227b9 (127.1.127.193:35151)
W20260807 15:02:39.139257 11953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:39.139493 11953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:39.139520 11953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:39.140734 11953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:39.140784 11953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:39.140816 11953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:39.142161 11953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:39.142385 11953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:39.142706 11953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:39.148069 11958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:39.148134 11961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:39.148331 11953 server_base.cc:1034] running on GCE node
W20260807 15:02:39.148772 11959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:39.149091 11953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:39.149398 11953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:39.150602 11953 hybrid_clock.cc:648] HybridClock initialized: now 1786114959150587 us; error 48 us; skew 500 ppm
I20260807 15:02:39.152299 11953 webserver.cc:460] Webserver started at http://127.1.127.194:44753/ using document root <none> and password file <none>
I20260807 15:02:39.152629 11953 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:39.152707 11953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:39.152876 11953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:39.154055 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "e1f706e8fcec4920a09bb6921b11792f"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "95d4a7b5d734e88e812580f33dbc11b6"
server_key_iv: "29cfc6798bbe3ac2d0608a65dcb43d82"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.154450 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "e1f706e8fcec4920a09bb6921b11792f"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "95d4a7b5d734e88e812580f33dbc11b6"
server_key_iv: "29cfc6798bbe3ac2d0608a65dcb43d82"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.156148 11953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:02:39.157148 11967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.157387 11953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:39.157512 11953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "e1f706e8fcec4920a09bb6921b11792f"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "95d4a7b5d734e88e812580f33dbc11b6"
server_key_iv: "29cfc6798bbe3ac2d0608a65dcb43d82"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.157614 11953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:39.161535 11953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:39.161890 11953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:39.162101 11953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:39.162339 11953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:39.162724 11953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:39.162770 11953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.162811 11953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:39.162837 11953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.169184 11953 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41993
I20260807 15:02:39.169226 12085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41993 every 8 connection(s)
I20260807 15:02:39.169580 11953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:39.172732 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.170359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:49569 (local address 127.1.127.254:34113)
0807 15:02:39.170444 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:39.170446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:39.170649 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:02:39.171226 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:39.171227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:39.171233 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:39.171311 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:39.172487 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:39.172488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:39.172489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:39.172493 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:39.172509 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:39.172514 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:39.172602 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:39.172660 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 15:02:39.172672 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:02:39.172955 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.170303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34113 (local address 127.1.127.194:49569)
0807 15:02:39.170513 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:39.170555 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:39.171105 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:39.171361 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:39.171395 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:39.171405 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:39.172329 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:39.172329 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:39.172359 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:39.172391 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:39.172600 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:39.172602 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:39.172605 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:02:39.172659 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:02:39.172711 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260807 15:02:39.173669 12086 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34113
I20260807 15:02:39.173789 12086 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:39.173964 12086 heartbeater.cc:511] Master 127.1.127.254:34113 requested a full tablet report, sending...
I20260807 15:02:39.174451 11753 ts_manager.cc:194] Registered new tserver with Master: e1f706e8fcec4920a09bb6921b11792f (127.1.127.194:41993)
I20260807 15:02:39.178073 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 11953
I20260807 15:02:39.178170 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:39.178391 1535 external_mini_cluster.cc:1428] Setting key bffe8d9ffd1ec2a4ab0faad917963b9c
I20260807 15:02:39.179693 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--builtin_ntp_servers=127.1.127.212:33401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:02:39.265082 12090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:39.265354 12090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:39.265394 12090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:39.266870 12090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:39.266953 12090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:39.267004 12090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:39.268738 12090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:39.269060 12090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:39.269399 12090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:39.274911 12096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:39.275017 12090 server_base.cc:1034] running on GCE node
W20260807 15:02:39.274911 12095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:39.274986 12098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:39.275492 12090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:39.275795 12090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:39.276954 12090 hybrid_clock.cc:648] HybridClock initialized: now 1786114959276919 us; error 52 us; skew 500 ppm
I20260807 15:02:39.278431 12090 webserver.cc:460] Webserver started at http://127.1.127.195:39235/ using document root <none> and password file <none>
I20260807 15:02:39.278802 12090 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:39.278874 12090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:39.279019 12090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:39.280169 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "d27174f9dd0047a48a740f7554afd487"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f56190715402abaf0b321b87848c4c5"
server_key_iv: "ca36818adcb7532f139d3612ea6837e4"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.280560 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "d27174f9dd0047a48a740f7554afd487"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f56190715402abaf0b321b87848c4c5"
server_key_iv: "ca36818adcb7532f139d3612ea6837e4"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.282085 12090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:02:39.283151 12104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.283352 12090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:02:39.283495 12090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "d27174f9dd0047a48a740f7554afd487"
format_stamp: "Formatted at 2026-08-07 15:02:39 on dist-test-slave-8tdl"
server_key: "8f56190715402abaf0b321b87848c4c5"
server_key_iv: "ca36818adcb7532f139d3612ea6837e4"
server_key_version: "encryptionkey@0"
I20260807 15:02:39.283584 12090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:39.287331 12090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:39.287621 12090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:39.287758 12090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:39.287966 12090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:39.288286 12090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:39.288339 12090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.288379 12090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:39.288405 12090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:39.294402 12090 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:43115
I20260807 15:02:39.294479 12224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:43115 every 8 connection(s)
I20260807 15:02:39.294973 12090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:39.295624 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 12090
I20260807 15:02:39.295717 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:39.295919 1535 external_mini_cluster.cc:1428] Setting key a57c332d3f6a0090da990b925262eeef
I20260807 15:02:39.298723 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.296224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55971 (local address 127.1.127.254:34113)
0807 15:02:39.296301 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:39.296303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:39.296314 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:02:39.297067 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:39.297068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:39.297074 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:39.297115 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:39.298415 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:39.298416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:39.298416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:02:39.298422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:39.298438 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:02:39.298442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:39.298524 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:39.298605 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:02:39.298623 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:02:39.298920 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:39.295929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:34113 (local address 127.1.127.195:55971)
0807 15:02:39.296164 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:39.296214 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:39.296970 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:39.297196 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:39.297200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:39.297210 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:39.298249 (+ 1039us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:02:39.298250 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:39.298280 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:02:39.298311 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:39.298522 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:39.298529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:39.298533 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:39.298606 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:02:39.298697 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260807 15:02:39.299438 12225 heartbeater.cc:347] Connected to a master server at 127.1.127.254:34113
I20260807 15:02:39.299557 12225 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:39.299713 12225 heartbeater.cc:511] Master 127.1.127.254:34113 requested a full tablet report, sending...
I20260807 15:02:39.300150 11753 ts_manager.cc:194] Registered new tserver with Master: d27174f9dd0047a48a740f7554afd487 (127.1.127.195:43115)
I20260807 15:02:39.308312 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:02:39.310644 11796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:02:39.310104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.1.127.254:34113)
0807 15:02:39.310200 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:39.310202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:39.310276 (+ 74us) server_negotiation.cc:407] Connection header received
0807 15:02:39.310326 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:39.310327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:39.310331 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:39.310381 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:39.310554 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34140: BlockingRecv error: recv got EOF from 127.0.0.1:34140 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:02:39.310715 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11816
I20260807 15:02:39.319478 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11953
I20260807 15:02:39.325682 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 12090
I20260807 15:02:39.331030 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 11721
2026-08-07T15:02:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (589 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:39 dist-test-slave-8tdl krb5kdc[12235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:39 dist-test-slave-8tdl krb5kdc[12235](info): set up 2 sockets
Aug 07 15:02:39 dist-test-slave-8tdl krb5kdc[12235](info): commencing operation
krb5kdc: starting...
W20260807 15:02:40.424264 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:40 dist-test-slave-8tdl krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:02:40.571802 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40083
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:40083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:40.665571 12250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:40.665918 12250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:40.665971 12250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:40.667502 12250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:02:40.667603 12250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:40.667634 12250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:40.667657 12250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:02:40.667695 12250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:02:40.669593 12250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:40083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:40.669945 12250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:40.670236 12250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:40.676105 12257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:40.676134 12259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:40.676151 12250 server_base.cc:1034] running on GCE node
W20260807 15:02:40.676116 12256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:40.676648 12250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:40.676994 12250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:40.678187 12250 hybrid_clock.cc:648] HybridClock initialized: now 1786114960678169 us; error 37 us; skew 500 ppm
Aug 07 15:02:40 dist-test-slave-8tdl krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114960, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:40.681239 12250 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:02:40.681778 12250 webserver.cc:460] Webserver started at http://127.1.127.254:34085/ using document root <none> and password file <none>
I20260807 15:02:40.682062 12250 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:40.682116 12250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:40.682264 12250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:40.683547 12250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "627f981fd807462f80bce08ecbc0111f"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "2e31ecb09d3c118945c0de96e9bad00e"
server_key_iv: "2fb3e7a51f55a4e256b17c0b2fc8f571"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.683959 12250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "627f981fd807462f80bce08ecbc0111f"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "2e31ecb09d3c118945c0de96e9bad00e"
server_key_iv: "2fb3e7a51f55a4e256b17c0b2fc8f571"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.685493 12250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:02:40.686517 12266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:40.686805 12250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:40.687008 12250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "627f981fd807462f80bce08ecbc0111f"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "2e31ecb09d3c118945c0de96e9bad00e"
server_key_iv: "2fb3e7a51f55a4e256b17c0b2fc8f571"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.687110 12250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:40.707908 12250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:40.710831 12250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:40.711134 12250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:40.716010 12250 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:40083
I20260807 15:02:40.716253 12324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:40083 every 8 connection(s)
I20260807 15:02:40.716565 12250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:02:40.717674 12325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:40.719370 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 12250
I20260807 15:02:40.719532 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:02:40.719766 1535 external_mini_cluster.cc:1428] Setting key 041bc69ab7163ba36feaf4bcc390fa24
I20260807 15:02:40.720680 12325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: Bootstrap starting.
I20260807 15:02:40.721333 12325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:40.721705 12325 log.cc:824] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:40.722572 12325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: No bootstrap required, opened a new log
I20260807 15:02:40.724885 12325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } }
I20260807 15:02:40.725041 12325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:40.725064 12325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627f981fd807462f80bce08ecbc0111f, State: Initialized, Role: FOLLOWER
I20260807 15:02:40.725206 12325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } }
I20260807 15:02:40.725296 12325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:02:40.725332 12325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:02:40.725414 12325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:40.726217 12325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } }
I20260807 15:02:40.726359 12325 leader_election.cc:302] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 627f981fd807462f80bce08ecbc0111f; no voters:
I20260807 15:02:40.726591 12325 leader_election.cc:288] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:02:40.726627 12331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:40.726899 12325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:02:40.726934 12331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [term 1 LEADER]: Becoming Leader. State: Replica: 627f981fd807462f80bce08ecbc0111f, State: Running, Role: LEADER
I20260807 15:02:40.727180 12331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } }
I20260807 15:02:40.727581 12331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 627f981fd807462f80bce08ecbc0111f. Latest consensus state: current_term: 1 leader_uuid: "627f981fd807462f80bce08ecbc0111f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } } }
I20260807 15:02:40.727674 12331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [sys.catalog]: This master's current role is: LEADER
Aug 07 15:02:40 dist-test-slave-8tdl krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:40.727936 12340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:02:40.728479 12340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:02:40.728833 12333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "627f981fd807462f80bce08ecbc0111f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627f981fd807462f80bce08ecbc0111f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40083 } } }
I20260807 15:02:40.728986 12333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f [sys.catalog]: This master's current role is: LEADER
I20260807 15:02:40.731199 12340 catalog_manager.cc:1347] Generated new cluster ID: fd4dcabaaa44490d86ef0498260d16f4
I20260807 15:02:40.731259 12340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:02:40.733161 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:40.720457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.1.127.254:40083)
0807 15:02:40.720688 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:40.720697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:40.720726 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:02:40.721301 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:40.721309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:40.721366 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:40.721550 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:40.722336 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.723437 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.724588 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.725168 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.725334 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:40.728620 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:40.728626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:40.728626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:40.728642 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:40.731219 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:40.731749 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:40.731753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:40.731753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:40.731869 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:40.732137 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:40.732138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:40.732139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:40.732454 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:40.732582 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:40.732774 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:02:40.732918 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260807 15:02:40.744480 12340 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:02:40.745357 12340 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:02:40.756572 12340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: Generated new TSK 0
I20260807 15:02:40.756953 12340 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:02:40.798574 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:40.886230 12352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:40.886478 12352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:40.886507 12352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:40.887895 12352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:40.887959 12352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:40.887990 12352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:02:40.889701 12352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:40.889987 12352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:40.890295 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:40.896553 12360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:40.896565 12357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:40.896549 12358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:40.896899 12352 server_base.cc:1034] running on GCE node
I20260807 15:02:40.897123 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:40.897356 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:40.898524 12352 hybrid_clock.cc:648] HybridClock initialized: now 1786114960898491 us; error 46 us; skew 500 ppm
Aug 07 15:02:40 dist-test-slave-8tdl krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114960, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:40.901368 12352 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:02:40.901851 12352 webserver.cc:460] Webserver started at http://127.1.127.193:38859/ using document root <none> and password file <none>
I20260807 15:02:40.902091 12352 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:40.902139 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:40.902287 12352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:40.903487 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "5f128a57841741b99623193aa960cd20"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "eb30e0279f10eb31bc1f0b044c0ed36e"
server_key_iv: "3b045cb6e7d6384af43d82b18084b45b"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.903877 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "5f128a57841741b99623193aa960cd20"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "eb30e0279f10eb31bc1f0b044c0ed36e"
server_key_iv: "3b045cb6e7d6384af43d82b18084b45b"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.905325 12352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:02:40.906288 12368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:40.906526 12352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:40.906620 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "5f128a57841741b99623193aa960cd20"
format_stamp: "Formatted at 2026-08-07 15:02:40 on dist-test-slave-8tdl"
server_key: "eb30e0279f10eb31bc1f0b044c0ed36e"
server_key_iv: "3b045cb6e7d6384af43d82b18084b45b"
server_key_version: "encryptionkey@0"
I20260807 15:02:40.906726 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:40.925382 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:40.928190 12352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:40.928378 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:40.928660 12352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:40.929010 12352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:40.929075 12352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:40.929111 12352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:40.929138 12352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:40.935523 12352 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:44837
I20260807 15:02:40.935619 12486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:44837 every 8 connection(s)
I20260807 15:02:40.935981 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:02:40 dist-test-slave-8tdl krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114960, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:40.945381 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 12352
I20260807 15:02:40.945501 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:02:40.945837 1535 external_mini_cluster.cc:1428] Setting key c11aca0db53ac11b9635212e6624f944
I20260807 15:02:40.948859 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:40.936835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40083 (local address 127.1.127.193:38133)
0807 15:02:40.937168 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:40.937228 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:40.937857 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:40.938148 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:40.938153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:40.938917 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:40.939180 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:40.939187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.940048 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.940050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:40.941003 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:40.941005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.941780 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.941782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:40.941950 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:40.942627 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:40.942637 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:40.944738 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:40.947184 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:40.947198 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:40.947201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:40.947516 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:40.947780 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:40.947781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:40.947781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:40.947926 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:40.948237 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:40.948243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:40.948391 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 15:02:40.948489 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:02:40.948654 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":95,"threads_started":1}
I20260807 15:02:40.948884 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:40.936929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:38133 (local address 127.1.127.254:40083)
0807 15:02:40.937102 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:40.937105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:40.937343 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:02:40.937991 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:40.937992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:40.938007 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:40.938065 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:40.939323 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.939911 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.941129 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:40.941639 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:40.941760 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:40.944933 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:40.944936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:40.944936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:40.944945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:40.947057 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:40.947608 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:40.947610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:40.947610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:40.947688 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:40.948001 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:40.948002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:40.948003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:40.948165 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:40.948237 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:40.948649 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 15:02:40.948773 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:02:40.949677 12487 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40083
I20260807 15:02:40.949785 12487 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:40.950039 12487 heartbeater.cc:511] Master 127.1.127.254:40083 requested a full tablet report, sending...
I20260807 15:02:40.950757 12285 ts_manager.cc:194] Registered new tserver with Master: 5f128a57841741b99623193aa960cd20 (127.1.127.193:44837)
I20260807 15:02:40.951491 12285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:38133
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:02:40.978899 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:41.069244 12492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:41.069556 12492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:41.069607 12492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:41.071079 12492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:41.071157 12492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:41.071209 12492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:02:41.073180 12492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:41.073508 12492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:41.073839 12492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:41.081744 12500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:41.081768 12497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:41.081744 12498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:41.082496 12492 server_base.cc:1034] running on GCE node
I20260807 15:02:41.082839 12492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:41.083171 12492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:41.084362 12492 hybrid_clock.cc:648] HybridClock initialized: now 1786114961084345 us; error 31 us; skew 500 ppm
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114961, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:41.087664 12492 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:02:41.088342 12492 webserver.cc:460] Webserver started at http://127.1.127.194:34147/ using document root <none> and password file <none>
I20260807 15:02:41.088637 12492 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:41.088694 12492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:41.088846 12492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:41.090184 12492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "b06ee38528f74cf39ccb963fb8bd2e66"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "6db34eab331734d3c2a4a521a3a052d6"
server_key_iv: "d49e8fc7b1bbcdd6f6661ae251aee415"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.090639 12492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "b06ee38528f74cf39ccb963fb8bd2e66"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "6db34eab331734d3c2a4a521a3a052d6"
server_key_iv: "d49e8fc7b1bbcdd6f6661ae251aee415"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.092340 12492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:41.093186 12508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.093504 12492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:41.093597 12492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "b06ee38528f74cf39ccb963fb8bd2e66"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "6db34eab331734d3c2a4a521a3a052d6"
server_key_iv: "d49e8fc7b1bbcdd6f6661ae251aee415"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.093713 12492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:41.112581 12492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:41.115276 12492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:41.115459 12492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:41.115772 12492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:41.116150 12492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:41.116189 12492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.116238 12492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:41.116269 12492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.123478 12492 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:39337
I20260807 15:02:41.123525 12628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:39337 every 8 connection(s)
I20260807 15:02:41.123894 12492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:02:41.125669 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 12492
I20260807 15:02:41.125780 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:02:41.126001 1535 external_mini_cluster.cc:1428] Setting key 47996481193d1ef9e88e8f0b898a78fc
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114961, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:02:41.135700 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.124735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:35415 (local address 127.1.127.254:40083)
0807 15:02:41.124803 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.124805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.125008 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:02:41.125767 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.125769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.125792 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.125868 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:41.127345 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.127985 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.129108 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.129340 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.129431 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.131976 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:41.131978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:41.131978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:41.131988 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:41.133814 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.134472 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.134473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.134474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.134547 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.134910 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.134911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.134911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.135057 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:41.135139 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.135499 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.135605 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:02:41.135897 12631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.124678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40083 (local address 127.1.127.194:35415)
0807 15:02:41.124879 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.124921 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.125590 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.125984 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.125988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.126940 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:41.127229 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.127236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.128093 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.128094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.129004 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.129007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.129450 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.129451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.129586 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.130211 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:41.130220 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:41.131847 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:41.134027 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:41.134040 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:41.134043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:41.134324 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:41.134627 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:41.134627 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:41.134628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:41.134828 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:41.135129 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:41.135137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:41.135324 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.135458 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.135642 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260807 15:02:41.136519 12629 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40083
I20260807 15:02:41.136650 12629 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:41.136863 12629 heartbeater.cc:511] Master 127.1.127.254:40083 requested a full tablet report, sending...
I20260807 15:02:41.137324 12285 ts_manager.cc:194] Registered new tserver with Master: b06ee38528f74cf39ccb963fb8bd2e66 (127.1.127.194:39337)
I20260807 15:02:41.137952 12285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:35415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:02:41.161758 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:02:41.251324 12634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:02:41.251561 12634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:02:41.251588 12634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:02:41.252852 12634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:02:41.252905 12634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:02:41.252934 12634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:02:41.254503 12634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:02:41.254830 12634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:02:41.255101 12634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:02:41.260653 12639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:02:41.260661 12642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:41.260679 12634 server_base.cc:1034] running on GCE node
W20260807 15:02:41.260695 12640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:02:41.261060 12634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:02:41.261325 12634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:02:41.262506 12634 hybrid_clock.cc:648] HybridClock initialized: now 1786114961262480 us; error 39 us; skew 500 ppm
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114961, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:02:41.265269 12634 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:02:41.265816 12634 webserver.cc:460] Webserver started at http://127.1.127.195:45059/ using document root <none> and password file <none>
I20260807 15:02:41.266057 12634 fs_manager.cc:359] Metadata directory not provided
I20260807 15:02:41.266104 12634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:02:41.266235 12634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:02:41.267398 12634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "573539a9b8724d94a6138e6f6d785165"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "f779b92c56d3d36aa2f76a63c33de20a"
server_key_iv: "441da6e6e316640e1fe448779e4e6d30"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.267772 12634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "573539a9b8724d94a6138e6f6d785165"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "f779b92c56d3d36aa2f76a63c33de20a"
server_key_iv: "441da6e6e316640e1fe448779e4e6d30"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.269239 12634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:02:41.270253 12649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.270566 12634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:02:41.270725 12634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "573539a9b8724d94a6138e6f6d785165"
format_stamp: "Formatted at 2026-08-07 15:02:41 on dist-test-slave-8tdl"
server_key: "f779b92c56d3d36aa2f76a63c33de20a"
server_key_iv: "441da6e6e316640e1fe448779e4e6d30"
server_key_version: "encryptionkey@0"
I20260807 15:02:41.270824 12634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:02:41.295588 12634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:02:41.298089 12634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:02:41.298272 12634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:02:41.298574 12634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:02:41.298988 12634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:02:41.299046 12634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.299090 12634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:02:41.299117 12634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:02:41.305276 12634 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:34695
I20260807 15:02:41.305413 12768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:34695 every 8 connection(s)
I20260807 15:02:41.305701 12634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:02:41.308518 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 12634
I20260807 15:02:41.308634 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:02:41.308872 1535 external_mini_cluster.cc:1428] Setting key dd5393067cf9f94088dd4049e917c820
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114961, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:02:41.317060 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.306580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:36527 (local address 127.1.127.254:40083)
0807 15:02:41.306646 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.306648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.306972 (+ 324us) server_negotiation.cc:407] Connection header received
0807 15:02:41.307584 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.307586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.307600 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.307651 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:41.308677 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.309583 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.310971 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.311191 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.311261 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.313822 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:41.313824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:41.313824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:41.313831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:41.315477 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.315980 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.315981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.315981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.316045 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.316330 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.316332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.316332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.316470 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:41.316552 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.316867 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.316945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:02:41.317162 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.306549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40083 (local address 127.1.127.195:36527)
0807 15:02:41.306766 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.306806 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.307466 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.307729 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.307733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.308319 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:02:41.308528 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.308533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.309711 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.309712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.310878 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.310881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.311297 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.311298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.311434 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.312024 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:02:41.312033 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:02:41.313704 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:02:41.315574 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:41.315582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:41.315584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:41.315877 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:41.316118 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:02:41.316118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:02:41.316119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:02:41.316235 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:02:41.316585 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:02:41.316592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:02:41.316722 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.316823 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.316956 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":147,"threads_started":1}
I20260807 15:02:41.317845 12769 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40083
I20260807 15:02:41.317958 12769 heartbeater.cc:464] Registering TS with master...
I20260807 15:02:41.318158 12769 heartbeater.cc:511] Master 127.1.127.254:40083 requested a full tablet report, sending...
I20260807 15:02:41.318763 12285 ts_manager.cc:194] Registered new tserver with Master: 573539a9b8724d94a6138e6f6d785165 (127.1.127.195:34695)
I20260807 15:02:41.319255 12285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:36527
I20260807 15:02:41.321267 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:02:41.329239 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.322730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.1.127.254:40083)
0807 15:02:41.322791 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.322793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.322890 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:02:41.322958 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.322959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.322970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.323032 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:02:41.323582 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.324050 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.324813 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.325005 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.325063 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.326295 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:02:41.326296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:02:41.326296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:02:41.326302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:02:41.328047 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.328482 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.328483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.328484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.328538 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:02:41.328724 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:02:41.328724 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:02:41.328725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:02:41.328916 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:02:41.329000 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.329086 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.329148 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:02:41.330986 12285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:02:41.332573 12285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:02:41.339956 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.1.127.194:39337)
0807 15:02:41.336176 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.336181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.336336 (+ 155us) server_negotiation.cc:407] Connection header received
0807 15:02:41.336380 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.336382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.336395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.336477 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:41.336693 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337721 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339006 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.339501 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339601 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.339674 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.339818 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.339862 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260807 15:02:41.339958 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39337 (local address 127.0.0.1:46732)
0807 15:02:41.336221 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.336231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.336278 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.336559 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.336560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.336562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:41.336601 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.336602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337830 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.337831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.338908 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.338912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.339603 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.339692 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.339708 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.339788 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.339848 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":76,"threads_started":1}
I20260807 15:02:41.340278 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.1.127.193:44837)
0807 15:02:41.335999 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.336002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.336206 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:02:41.336261 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.336263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.336280 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.336362 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:41.336469 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337269 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.338905 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.339503 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339604 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.339673 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.340144 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.340200 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":84,"threads_started":1}
I20260807 15:02:41.339959 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:44837 (local address 127.0.0.1:51484)
0807 15:02:41.336073 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.336084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.336138 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.336359 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.336359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.336361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:41.336402 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.336403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337426 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.337427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.338816 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.338818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.339609 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.339700 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.339709 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.339811 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.339861 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":113,"threads_started":1}
I20260807 15:02:41.341009 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:34695 (local address 127.0.0.1:60364)
0807 15:02:41.335715 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.335763 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.335840 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.336098 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.336101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.336104 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:41.336183 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.336186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337626 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.337627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.338815 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.338818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.340709 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.340710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.340782 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.340788 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.340843 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.340889 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":7,"thread_start_us":78,"threads_started":1}
I20260807 15:02:41.340850 12559 tablet_service.cc:1467] Processing CreateTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 (DEFAULT_TABLE table=test-table [id=9188f5e7b8b444f3b34203816b28b67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:41.341300 12559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c69aa7d021441a7abfe542ec97e2616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:41.341321 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.335670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.1.127.195:34695)
0807 15:02:41.335893 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.335898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.335908 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:02:41.335951 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.335953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.335969 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.336276 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:41.336345 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.337486 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.339715 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.340623 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.340705 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.340759 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.341190 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.341241 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
I20260807 15:02:41.341444 12419 tablet_service.cc:1467] Processing CreateTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 (DEFAULT_TABLE table=test-table [id=9188f5e7b8b444f3b34203816b28b67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:41.341717 12419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c69aa7d021441a7abfe542ec97e2616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:41.343449 12701 tablet_service.cc:1467] Processing CreateTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 (DEFAULT_TABLE table=test-table [id=9188f5e7b8b444f3b34203816b28b67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:02:41.343824 12701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c69aa7d021441a7abfe542ec97e2616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:02:41.344781 12786 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: Bootstrap starting.
I20260807 15:02:41.345148 12787 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Bootstrap starting.
I20260807 15:02:41.345518 12786 tablet_bootstrap.cc:654] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:41.345858 12787 tablet_bootstrap.cc:654] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:41.346128 12786 log.cc:824] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:41.346496 12787 log.cc:824] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:41.347018 12786 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: No bootstrap required, opened a new log
I20260807 15:02:41.347093 12786 ts_tablet_manager.cc:1397] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:02:41.347321 12787 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: No bootstrap required, opened a new log
I20260807 15:02:41.347396 12787 ts_tablet_manager.cc:1397] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:02:41.347739 12790 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Bootstrap starting.
I20260807 15:02:41.348490 12790 tablet_bootstrap.cc:654] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Neither blocks nor log segments found. Creating new log.
I20260807 15:02:41.348862 12786 raft_consensus.cc:348] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.349035 12786 raft_consensus.cc:374] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:41.349082 12786 raft_consensus.cc:729] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f128a57841741b99623193aa960cd20, State: Initialized, Role: FOLLOWER
I20260807 15:02:41.349087 12790 log.cc:824] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Log is configured to *not* fsync() on all Append() calls
I20260807 15:02:41.349226 12786 consensus_queue.cc:260] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.349563 12786 ts_tablet_manager.cc:1428] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:41.349613 12487 heartbeater.cc:503] Master 127.1.127.254:40083 was elected leader, sending a full tablet report...
I20260807 15:02:41.349694 12787 raft_consensus.cc:348] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.349902 12787 raft_consensus.cc:374] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:41.349949 12787 raft_consensus.cc:729] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06ee38528f74cf39ccb963fb8bd2e66, State: Initialized, Role: FOLLOWER
I20260807 15:02:41.350086 12787 consensus_queue.cc:260] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.350281 12790 tablet_bootstrap.cc:492] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: No bootstrap required, opened a new log
I20260807 15:02:41.350359 12790 ts_tablet_manager.cc:1397] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:02:41.350414 12787 ts_tablet_manager.cc:1428] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:02:41.350567 12629 heartbeater.cc:503] Master 127.1.127.254:40083 was elected leader, sending a full tablet report...
I20260807 15:02:41.352106 12790 raft_consensus.cc:348] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.352248 12790 raft_consensus.cc:374] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:02:41.352272 12790 raft_consensus.cc:729] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573539a9b8724d94a6138e6f6d785165, State: Initialized, Role: FOLLOWER
I20260807 15:02:41.352402 12790 consensus_queue.cc:260] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.352658 12790 ts_tablet_manager.cc:1428] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:02:41.352762 12769 heartbeater.cc:503] Master 127.1.127.254:40083 was elected leader, sending a full tablet report...
W20260807 15:02:41.374509 12630 tablet.cc:2369] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:02:41.385515 12793 raft_consensus.cc:482] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:02:41.385653 12793 raft_consensus.cc:504] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.386215 12793 leader_election.cc:288] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 573539a9b8724d94a6138e6f6d785165 (127.1.127.195:34695), 5f128a57841741b99623193aa960cd20 (127.1.127.193:44837)
I20260807 15:02:41.389699 12631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.386430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:44837 (local address 127.1.127.194:60669)
0807 15:02:41.386501 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.386515 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.386567 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.386765 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.386766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.386768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:41.386851 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.386852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.387738 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.387739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.388637 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.388640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.389378 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.389379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.389504 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.389509 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.389572 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.389613 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:02:41.389696 12782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.386520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:60669 (local address 127.1.127.193:44837)
0807 15:02:41.386593 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.386595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.386626 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:02:41.386655 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.386656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.386664 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.386757 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:41.386965 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.387642 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.388738 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.389299 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.389371 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.389390 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.389600 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.389643 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:02:41.390156 12439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c69aa7d021441a7abfe542ec97e2616" candidate_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f128a57841741b99623193aa960cd20" is_pre_election: true
I20260807 15:02:41.390219 12783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.386501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:47621 (local address 127.1.127.195:34695)
0807 15:02:41.386572 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.386574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.386914 (+ 340us) server_negotiation.cc:407] Connection header received
0807 15:02:41.387091 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.387095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.387104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.387166 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:02:41.387350 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.387937 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.389123 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.389691 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.389761 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.389769 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.390115 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.390157 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:02:41.390231 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.386433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:34695 (local address 127.1.127.194:47621)
0807 15:02:41.386824 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.386833 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:02:41.386967 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:02:41.387188 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:02:41.387189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:02:41.387191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:02:41.387266 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.387268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.388090 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.388091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.389026 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:02:41.389029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.389937 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.389938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:02:41.390021 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.390025 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:02:41.390106 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:02:41.390146 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":190,"thread_start_us":139,"threads_started":1}
I20260807 15:02:41.390364 12439 raft_consensus.cc:2454] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b06ee38528f74cf39ccb963fb8bd2e66 in term 0.
I20260807 15:02:41.390630 12511 leader_election.cc:302] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f128a57841741b99623193aa960cd20, b06ee38528f74cf39ccb963fb8bd2e66; no voters:
I20260807 15:02:41.390600 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c69aa7d021441a7abfe542ec97e2616" candidate_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "573539a9b8724d94a6138e6f6d785165" is_pre_election: true
I20260807 15:02:41.390803 12721 raft_consensus.cc:2454] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b06ee38528f74cf39ccb963fb8bd2e66 in term 0.
I20260807 15:02:41.390805 12793 raft_consensus.cc:2781] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:02:41.390882 12793 raft_consensus.cc:482] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:02:41.390923 12793 raft_consensus.cc:3037] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:41.391765 12793 raft_consensus.cc:504] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.391948 12793 leader_election.cc:288] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [CANDIDATE]: Term 1 election: Requested vote from peers 573539a9b8724d94a6138e6f6d785165 (127.1.127.195:34695), 5f128a57841741b99623193aa960cd20 (127.1.127.193:44837)
I20260807 15:02:41.392122 12439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c69aa7d021441a7abfe542ec97e2616" candidate_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f128a57841741b99623193aa960cd20"
I20260807 15:02:41.392206 12439 raft_consensus.cc:3037] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:41.392218 12721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c69aa7d021441a7abfe542ec97e2616" candidate_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "573539a9b8724d94a6138e6f6d785165"
I20260807 15:02:41.392304 12721 raft_consensus.cc:3037] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:02:41.393057 12439 raft_consensus.cc:2454] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06ee38528f74cf39ccb963fb8bd2e66 in term 1.
I20260807 15:02:41.393225 12721 raft_consensus.cc:2454] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06ee38528f74cf39ccb963fb8bd2e66 in term 1.
I20260807 15:02:41.393237 12511 leader_election.cc:302] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f128a57841741b99623193aa960cd20, b06ee38528f74cf39ccb963fb8bd2e66; no voters:
I20260807 15:02:41.393319 12793 raft_consensus.cc:2781] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:02:41.393513 12793 raft_consensus.cc:686] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 1 LEADER]: Becoming Leader. State: Replica: b06ee38528f74cf39ccb963fb8bd2e66, State: Running, Role: LEADER
I20260807 15:02:41.393622 12793 consensus_queue.cc:237] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } }
I20260807 15:02:41.394582 12284 catalog_manager.cc:5495] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to b06ee38528f74cf39ccb963fb8bd2e66 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b06ee38528f74cf39ccb963fb8bd2e66" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39337 } health_report { overall_health: HEALTHY } } }
I20260807 15:02:41.415999 12785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.412958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.1.127.194:39337)
0807 15:02:41.413046 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 15:02:41.413048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:02:41.413060 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:02:41.413154 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:02:41.413155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:02:41.413177 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:02:41.413255 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:02:41.413445 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.414033 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.414996 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:02:41.415201 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:02:41.415266 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:02:41.415506 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:02:41.415620 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:02:41.415683 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:02:41.415737 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:02:41.415756 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:02:41.418021 12439 raft_consensus.cc:1261] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20 [term 1 FOLLOWER]: Refusing update from remote peer b06ee38528f74cf39ccb963fb8bd2e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:41.418042 12721 raft_consensus.cc:1261] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 1 FOLLOWER]: Refusing update from remote peer b06ee38528f74cf39ccb963fb8bd2e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:02:41.418426 12793 consensus_queue.cc:1035] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f128a57841741b99623193aa960cd20" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 44837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:41.418635 12796 consensus_queue.cc:1035] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "573539a9b8724d94a6138e6f6d785165" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:02:41.421613 12800 mvcc.cc:205] Tried to move back new op lower bound from 7315926881965146112 to 7315926881868873728. Current Snapshot: MvccSnapshot[applied={T|T < 7315926881965146112}]
I20260807 15:02:41.421972 12802 mvcc.cc:205] Tried to move back new op lower bound from 7315926881965146112 to 7315926881868873728. Current Snapshot: MvccSnapshot[applied={T|T < 7315926881965146112}]
I20260807 15:02:41.422559 12801 mvcc.cc:205] Tried to move back new op lower bound from 7315926881965146112 to 7315926881868873728. Current Snapshot: MvccSnapshot[applied={T|T < 7315926881965146112}]
I20260807 15:02:41.424196 12284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:41.424335 12284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:02:41.425515 12284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 627f981fd807462f80bce08ecbc0111f: Sending DeleteTablet for 3 replicas of tablet 5c69aa7d021441a7abfe542ec97e2616
I20260807 15:02:41.426012 12701 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:41 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:60364
I20260807 15:02:41.426021 12559 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:41 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46732
I20260807 15:02:41.426340 12810 tablet_replica.cc:330] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: stopping tablet replica
I20260807 15:02:41.426319 12419 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c69aa7d021441a7abfe542ec97e2616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:02:41 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:51484
I20260807 15:02:41.426479 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 12352
I20260807 15:02:41.426630 12812 tablet_replica.cc:330] T 5c69aa7d021441a7abfe542ec97e2616 P 5f128a57841741b99623193aa960cd20: stopping tablet replica
I20260807 15:02:41.426618 12810 raft_consensus.cc:2229] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:02:41.426885 12810 raft_consensus.cc:2258] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:41.427603 12810 ts_tablet_manager.cc:1905] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:41.429816 12810 ts_tablet_manager.cc:1918] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:41.429975 12810 log.cc:1196] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/5c69aa7d021441a7abfe542ec97e2616
I20260807 15:02:41.430390 12810 ts_tablet_manager.cc:1939] T 5c69aa7d021441a7abfe542ec97e2616 P b06ee38528f74cf39ccb963fb8bd2e66: Deleting consensus metadata
I20260807 15:02:41.430953 12270 catalog_manager.cc:4841] TS b06ee38528f74cf39ccb963fb8bd2e66 (127.1.127.194:39337): tablet 5c69aa7d021441a7abfe542ec97e2616 (table test-table [id=9188f5e7b8b444f3b34203816b28b67c]) successfully deleted
I20260807 15:02:41.432138 12811 tablet_replica.cc:330] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: stopping tablet replica
I20260807 15:02:41.432314 12811 raft_consensus.cc:2229] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:02:41.432471 12811 raft_consensus.cc:2258] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:02:41.435467 12811 ts_tablet_manager.cc:1905] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:02:41.437584 12811 ts_tablet_manager.cc:1918] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:02:41.437703 12811 log.cc:1196] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/5c69aa7d021441a7abfe542ec97e2616
W20260807 15:02:41.437727 12269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:44837 (error 108)
I20260807 15:02:41.438194 12811 ts_tablet_manager.cc:1939] T 5c69aa7d021441a7abfe542ec97e2616 P 573539a9b8724d94a6138e6f6d785165: Deleting consensus metadata
I20260807 15:02:41.438395 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:02:41.438257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:44837 (local address 127.0.0.1:51490)
0807 15:02:41.438315 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:02:41.438353 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:02:41.438522 12269 catalog_manager.cc:4584] TS 5f128a57841741b99623193aa960cd20 (127.1.127.193:44837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c69aa7d021441a7abfe542ec97e2616: Network error: Client connection negotiation failed: client connection to 127.1.127.193:44837: connect: Connection refused (error 111)
I20260807 15:02:41.438849 12270 catalog_manager.cc:4841] TS 573539a9b8724d94a6138e6f6d785165 (127.1.127.195:34695): tablet 5c69aa7d021441a7abfe542ec97e2616 (table test-table [id=9188f5e7b8b444f3b34203816b28b67c]) successfully deleted
I20260807 15:02:41.439028 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 12492
I20260807 15:02:41.448406 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 12634
I20260807 15:02:41.455796 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 12250
2026-08-07T15:02:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2126 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12816](info): set up 2 sockets
Aug 07 15:02:41 dist-test-slave-8tdl krb5kdc[12816](info): commencing operation
krb5kdc: starting...
W20260807 15:02:42.545128 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:02:42 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786114962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:02:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangeradmin_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangeradmin_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangerlookup_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangerlookup_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
I20260807 15:02:42.750587 1535 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbmaIkX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/postgres -l logfile start
2026-08-07 15:02:45.989 UTC [12848] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:02:45.989 UTC [12848] LOG: listening on IPv4 address "127.1.127.212", port 42227
2026-08-07 15:02:45.993 UTC [12848] LOG: listening on Unix socket "/tmp/.s.PGSQL.42227"
2026-08-07 15:02:46.020 UTC [12851] LOG: database system was shut down at 2026-08-07 15:02:43 UTC
2026-08-07 15:02:46.026 UTC [12848] LOG: database system is ready to accept connections
I20260807 15:02:47.021853 1535 mini_postgres.cc:96] Postgres bound to 42227
2026-08-07 15:02:47.029 UTC [12859] FATAL: database "slave" does not exist
127.1.127.212:42227 - accepting connections
I20260807 15:02:47.030942 1535 mini_ranger.cc:161] Starting Ranger...
I20260807 15:02:47.049280 1535 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:02:47.725167 1535 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:02:47.725258 1535 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin
2026-08-07 15:02:48,097 [I] DB FLAVOR :POSTGRES
2026-08-07 15:02:48,098 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:02:48,098 [I] Checking connection..
2026-08-07 15:02:48,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:02:48,384 [I] Checking connection passed.
2026-08-07 15:02:48,384 [I] --------- Verifying version history table ---------
2026-08-07 15:02:48,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:02:48,646 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:02:48,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:02:48,923 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:02:48,923 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:02:48,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:02:49.218 UTC [12925] WARNING: there is no transaction in progress
2026-08-07 15:02:49,227 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:02:49,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:02:49,495 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:02:49,495 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:02:49,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:02:49,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:02:50,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-07 15:02:50,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:02:51,011 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:02:51,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:02:51,317 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:02:51,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:02:51,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:02:51,830 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:02:51,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:02:53.055 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.076 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.098 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.122 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.148 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.174 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.259 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.269 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.290 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.305 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.320 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.339 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.352 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.361 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.371 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:53.381 UTC [13038] WARNING: there is no transaction in progress
2026-08-07 15:02:54,423 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:02:54,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:02:54,690 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:02:54,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:02:54,968 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:02:54,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:02:55,249 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:02:55,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:02:55,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:02:55,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-07 15:02:56,064 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:02:56,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:02:56,327 [I] DB_PATCHES have already been applied
I20260807 15:02:56.333264 1535 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbmaIkX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:02:56.333344 1535 mini_ranger.cc:193] Using host: 127.1.127.212
I20260807 15:02:56.337459 1535 mini_ranger.cc:227] Ranger admin URL: http://127.1.127.212:50999
Aug 07, 2026 3:02:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:02:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:02:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:02:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:02:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:02:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:02:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:02:57.520028 1535 mini_ranger.cc:160] Time spent starting Ranger: real 10.489s user 0.000s sys 0.013s
Aug 07, 2026 3:03:19 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:03:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:03:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:03:19 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:03:20 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:03:21.388863 1535 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangerkms_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/rangerkms_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.127.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.127.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:03:21 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115001, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:03:21.511089 1535 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:03:21.524204 1535 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:03:22.807796 1535 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:03:22.807888 1535 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms
I20260807 15:03:22.807911 1535 mini_ranger_kms.cc:227] Using postgres at 127.1.127.212:42227
2026-08-07 15:03:23,062 [I] DB FLAVOR :POSTGRES
2026-08-07 15:03:23,062 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:03:23,062 [I] Checking connection
2026-08-07 15:03:23,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:03:23,322 [I] connection success
2026-08-07 15:03:23,323 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:03:23,323 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:03:23,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:03:23,596 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:03:23,596 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:03:23,597 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:03:23,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:42227/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:03:23,922 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:03:24.538163 1535 mini_ranger_kms.cc:310] Created kms service
I20260807 15:03:24.761932 1535 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:03:24.833228 1535 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:03:24 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115001, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:03:25.364342 1535 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:03:25.364538 1535 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:03:25.364562 1535 mini_ranger_kms.cc:240] Using host: 127.1.127.212
I20260807 15:03:25.369971 1535 mini_ranger_kms.cc:276] Ranger KMS PID: 13229
I20260807 15:03:25.370162 1535 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.127.212:59365
Aug 07, 2026 3:03:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:03:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:03:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:03:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:03:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:03:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 15:03:27.836318 1535 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.325s user 0.000s sys 0.238s
I20260807 15:03:27.836438 1535 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:03:27.836520 1535 mini_ranger_kms.cc:401] 127.1.127.212:59365/kms/v1/keys
Aug 07 15:03:28 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115008, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 15:03:28 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115008, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
Aug 07 15:03:28 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115008, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
Aug 07 15:03:31 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:03:31.627928 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46075
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:46075
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:59365/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:03:31.715379 13282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:03:31.715641 13282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:03:31.715682 13282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:03:31.717010 13282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:03:31.717095 13282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:03:31.717119 13282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:03:31.717152 13282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:03:31.717180 13282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:03:31.719009 13282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.127.212:59365/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:46075
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:03:31.719313 13282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:03:31.719566 13282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:03:31.727025 13288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:03:31.727025 13290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:31.727106 13282 server_base.cc:1034] running on GCE node
W20260807 15:03:31.727166 13287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:31.727628 13282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:03:31.728083 13282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:03:31.729264 13282 hybrid_clock.cc:648] HybridClock initialized: now 1786115011729249 us; error 33 us; skew 500 ppm
Aug 07 15:03:31 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115011, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:03:31.738973 13282 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:03:31.739558 13282 webserver.cc:460] Webserver started at http://127.1.127.254:39765/ using document root <none> and password file <none>
I20260807 15:03:31.739807 13282 fs_manager.cc:359] Metadata directory not provided
I20260807 15:03:31.739873 13282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:03:31.740046 13282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:03:31 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115011, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:03:31.812294 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "eedcab2940fe44ff8544fa62ab8e1920"
format_stamp: "Formatted at 2026-08-07 15:03:31 on dist-test-slave-8tdl"
server_key: "f686ad2220c3efb839b669cbde5a21d9"
server_key_iv: "74fc655d88ddd3c8cb671a2871677c52"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:31.812824 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "eedcab2940fe44ff8544fa62ab8e1920"
format_stamp: "Formatted at 2026-08-07 15:03:31 on dist-test-slave-8tdl"
server_key: "f686ad2220c3efb839b669cbde5a21d9"
server_key_iv: "74fc655d88ddd3c8cb671a2871677c52"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:31.814844 13282 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:03:31.829316 13297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:31.829636 13282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:03:31.829747 13282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "eedcab2940fe44ff8544fa62ab8e1920"
format_stamp: "Formatted at 2026-08-07 15:03:31 on dist-test-slave-8tdl"
server_key: "f686ad2220c3efb839b669cbde5a21d9"
server_key_iv: "74fc655d88ddd3c8cb671a2871677c52"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:31.829905 13282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:03:31.861449 13282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:03:31.864082 13282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:03:31.864275 13282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:03:31.870214 13282 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:46075
I20260807 15:03:31.870240 13351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:46075 every 8 connection(s)
I20260807 15:03:31.870728 13282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:03:31.871639 13352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:03:31.875221 13352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: Bootstrap starting.
I20260807 15:03:31.876003 13352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: Neither blocks nor log segments found. Creating new log.
I20260807 15:03:31.876341 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 13282
I20260807 15:03:31.876381 13352 log.cc:824] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: Log is configured to *not* fsync() on all Append() calls
I20260807 15:03:31.876472 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:03:31.876688 1535 external_mini_cluster.cc:1428] Setting key dcac87080ae9c592139c43e1f4700bf3
I20260807 15:03:31.877539 13352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: No bootstrap required, opened a new log
I20260807 15:03:31.881026 13352 raft_consensus.cc:348] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } }
I20260807 15:03:31.881232 13352 raft_consensus.cc:374] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:03:31.881264 13352 raft_consensus.cc:729] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eedcab2940fe44ff8544fa62ab8e1920, State: Initialized, Role: FOLLOWER
I20260807 15:03:31.881405 13352 consensus_queue.cc:260] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } }
I20260807 15:03:31.881524 13352 raft_consensus.cc:388] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:03:31.881678 13352 raft_consensus.cc:482] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:03:31.881771 13352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:03:31.882531 13352 raft_consensus.cc:504] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } }
I20260807 15:03:31.882680 13352 leader_election.cc:302] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eedcab2940fe44ff8544fa62ab8e1920; no voters:
I20260807 15:03:31.882977 13352 leader_election.cc:288] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:03:31.883008 13357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:03:31.883296 13357 raft_consensus.cc:686] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [term 1 LEADER]: Becoming Leader. State: Replica: eedcab2940fe44ff8544fa62ab8e1920, State: Running, Role: LEADER
I20260807 15:03:31.883435 13352 sys_catalog.cc:564] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:03:31.883446 13357 consensus_queue.cc:237] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } }
I20260807 15:03:31.884370 13357 sys_catalog.cc:455] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eedcab2940fe44ff8544fa62ab8e1920. Latest consensus state: current_term: 1 leader_uuid: "eedcab2940fe44ff8544fa62ab8e1920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } } }
I20260807 15:03:31.884466 13357 sys_catalog.cc:458] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [sys.catalog]: This master's current role is: LEADER
I20260807 15:03:31.884867 13364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:03:31.885566 13364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:03:31 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:03:31.886242 13352 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:03:31.887131 13359 sys_catalog.cc:455] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eedcab2940fe44ff8544fa62ab8e1920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedcab2940fe44ff8544fa62ab8e1920" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46075 } } }
I20260807 15:03:31.887315 13359 sys_catalog.cc:458] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920 [sys.catalog]: This master's current role is: LEADER
I20260807 15:03:31.895516 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:31.877446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.1.127.254:46075)
0807 15:03:31.878009 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:31.878020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:31.878058 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:03:31.879270 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:31.879279 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:31.879348 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:31.879613 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:03:31.880333 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:31.881473 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:31.882707 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:31.883358 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:31.883834 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:31.886611 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:03:31.886616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:03:31.886617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:03:31.886630 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:03:31.892210 (+ 5580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:31.892905 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:31.892908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:31.892909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:31.893052 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:31.893299 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:31.893300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:31.893300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:31.893745 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:03:31.894531 (+ 786us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:31.894630 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:03:31.894879 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":88,"threads_started":1}
I20260807 15:03:31.897001 13364 catalog_manager.cc:1347] Generated new cluster ID: c3499dc05b7a4df4a6e0de843eb3fb12
I20260807 15:03:31.897068 13364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:03:31.915671 13364 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:03:31.916541 13364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:03:31.923058 13364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: Generated new TSK 0
I20260807 15:03:31.923347 13364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:03:32.703200 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:59365/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:03:32.818218 13401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:03:32.818547 13401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:03:32.818593 13401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:03:32.820116 13401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:03:32.820191 13401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:03:32.820242 13401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:03:32.821895 13401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.127.212:59365/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:03:32.822221 13401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:03:32.822508 13401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:03:32.828166 13407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:03:32.828167 13406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:32.828439 13401 server_base.cc:1034] running on GCE node
W20260807 15:03:32.828397 13409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:32.828691 13401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:03:32.829005 13401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:03:32.830209 13401 hybrid_clock.cc:648] HybridClock initialized: now 1786115012830194 us; error 36 us; skew 500 ppm
Aug 07 15:03:32 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115012, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:03:32.833629 13401 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:03:32.834275 13401 webserver.cc:460] Webserver started at http://127.1.127.193:34029/ using document root <none> and password file <none>
I20260807 15:03:32.834587 13401 fs_manager.cc:359] Metadata directory not provided
I20260807 15:03:32.834667 13401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:03:32.834909 13401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:03:32 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115012, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:03:32.850636 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "b332a07eeba448df96ad9c1f1d08a563"
format_stamp: "Formatted at 2026-08-07 15:03:32 on dist-test-slave-8tdl"
server_key: "61eec45c340b00dae584638bbe28ff1a"
server_key_iv: "278f2b165dd019225f66cd5a0af2a8bf"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:32.852288 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "b332a07eeba448df96ad9c1f1d08a563"
format_stamp: "Formatted at 2026-08-07 15:03:32 on dist-test-slave-8tdl"
server_key: "61eec45c340b00dae584638bbe28ff1a"
server_key_iv: "278f2b165dd019225f66cd5a0af2a8bf"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:32.856487 13401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260807 15:03:32.876797 13420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:32.877192 13401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:03:32.877353 13401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "b332a07eeba448df96ad9c1f1d08a563"
format_stamp: "Formatted at 2026-08-07 15:03:32 on dist-test-slave-8tdl"
server_key: "61eec45c340b00dae584638bbe28ff1a"
server_key_iv: "278f2b165dd019225f66cd5a0af2a8bf"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:32.877463 13401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:03:32.893108 13401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:03:32.895972 13401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:03:32.896181 13401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:03:32.896449 13401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:03:32.896919 13401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:03:32.897022 13401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:32.897141 13401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:03:32.897226 13401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:32.905854 13401 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:34703
I20260807 15:03:32.906029 13537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:34703 every 8 connection(s)
I20260807 15:03:32.906415 13401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:03:32.913481 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 13401
I20260807 15:03:32.913607 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:03:32.913929 1535 external_mini_cluster.cc:1428] Setting key 4bc4ee761e212af0cfae49a19402d530
Aug 07 15:03:32 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115012, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:03:32.925233 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:32.907285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46075 (local address 127.1.127.193:45195)
0807 15:03:32.907634 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:32.907685 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:32.908521 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:32.908954 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:32.908958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:32.909245 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:03:32.909558 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:32.909566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:32.910606 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:32.910607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:32.912025 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:32.912028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:32.912775 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:32.912777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:32.912959 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:32.914380 (+ 1421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:03:32.914392 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:03:32.919447 (+ 5055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:03:32.922649 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:32.922664 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:32.922668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:32.923288 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:32.923585 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:32.923586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:32.923587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:32.923761 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:32.924128 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:03:32.924137 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:03:32.924342 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 15:03:32.924570 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 15:03:32.924903 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":93,"threads_started":1}
I20260807 15:03:32.925935 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:32.907322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:45195 (local address 127.1.127.254:46075)
0807 15:03:32.907544 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:32.907547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:32.907776 (+ 229us) server_negotiation.cc:407] Connection header received
0807 15:03:32.908674 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:32.908676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:32.908692 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:32.908741 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:03:32.909683 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:32.910467 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:32.912147 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:32.912457 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:32.912614 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:32.919635 (+ 7021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:03:32.919638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:03:32.919638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:03:32.919650 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:03:32.922481 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:32.923401 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:32.923402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:32.923403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:32.923486 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:32.923847 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:32.923848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:32.923848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:32.924021 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:03:32.924094 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:32.925726 (+ 1632us) server_negotiation.cc:299] Negotiation successful
0807 15:03:32.925835 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":115,"threads_started":1}
I20260807 15:03:32.926875 13538 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46075
I20260807 15:03:32.926988 13538 heartbeater.cc:464] Registering TS with master...
I20260807 15:03:32.927205 13538 heartbeater.cc:511] Master 127.1.127.254:46075 requested a full tablet report, sending...
I20260807 15:03:32.927778 13314 ts_manager.cc:194] Registered new tserver with Master: b332a07eeba448df96ad9c1f1d08a563 (127.1.127.193:34703)
I20260807 15:03:32.928668 13314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:45195
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:03:32.958267 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:59365/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:03:33.103546 13544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:03:33.103964 13544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:03:33.104076 13544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:03:33.106264 13544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:03:33.106587 13544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:03:33.106746 13544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:03:33.109517 13544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.127.212:59365/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:03:33.110076 13544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:03:33.110461 13544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:03:33.118242 13552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:03:33.119238 13551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:33.119900 13544 server_base.cc:1034] running on GCE node
W20260807 15:03:33.119256 13554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:03:33.121141 13544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:03:33.121465 13544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:03:33.122653 13544 hybrid_clock.cc:648] HybridClock initialized: now 1786115013122625 us; error 37 us; skew 500 ppm
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:03:33.126483 13544 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:03:33.127140 13544 webserver.cc:460] Webserver started at http://127.1.127.194:35995/ using document root <none> and password file <none>
I20260807 15:03:33.127416 13544 fs_manager.cc:359] Metadata directory not provided
I20260807 15:03:33.127472 13544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:03:33.127619 13544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:03:33.150946 13544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "ee4ef6c0e984446cb6217c10be8520cc"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "df8ca77ff0f641c62271c88c1b69df61"
server_key_iv: "9f22b1fce53d56e9fc06bbaa306c8526"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.151786 13544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "ee4ef6c0e984446cb6217c10be8520cc"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "df8ca77ff0f641c62271c88c1b69df61"
server_key_iv: "9f22b1fce53d56e9fc06bbaa306c8526"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.163251 13544 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.001s sys 0.000s
I20260807 15:03:33.182120 13562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.182490 13544 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260807 15:03:33.182650 13544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "ee4ef6c0e984446cb6217c10be8520cc"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "df8ca77ff0f641c62271c88c1b69df61"
server_key_iv: "9f22b1fce53d56e9fc06bbaa306c8526"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.182793 13544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:03:33.199681 13544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:03:33.202198 13544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:03:33.202400 13544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:03:33.202714 13544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:03:33.203197 13544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:03:33.203254 13544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.203291 13544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:03:33.203372 13544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.211664 13544 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:39963
I20260807 15:03:33.212227 13544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:03:33.216034 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 13544
I20260807 15:03:33.216152 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:03:33.216373 1535 external_mini_cluster.cc:1428] Setting key f5a68d55dadc6bec085be2a63143f54b
I20260807 15:03:33.216667 13679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:39963 every 8 connection(s)
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:03:33.235355 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.217380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:41081 (local address 127.1.127.254:46075)
0807 15:03:33.217444 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.217447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.219676 (+ 2229us) server_negotiation.cc:407] Connection header received
0807 15:03:33.220439 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.220440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.220459 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.220493 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:03:33.221513 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.222151 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.225845 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.226207 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.226296 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.229866 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:03:33.229868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:03:33.229868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:03:33.229879 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:03:33.232018 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.232764 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.232765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.232766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.232834 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.233179 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.233181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.233181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.233711 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:03:33.233794 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.234895 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.235117 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
I20260807 15:03:33.241695 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.217361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46075 (local address 127.1.127.194:41081)
0807 15:03:33.219494 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.219564 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.220290 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.220733 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.220738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.221053 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:03:33.221374 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.221381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.222276 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.222277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.223531 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.223534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.226405 (+ 2871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.226407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.226594 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.227458 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:03:33.227471 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:03:33.229682 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:03:33.232162 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:33.232175 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:33.232179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:33.232639 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:33.232913 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:33.232914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:33.232914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:33.233086 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:33.234157 (+ 1071us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:03:33.234167 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:03:33.234501 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.240702 (+ 6201us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.241189 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2063,"thread_start_us":211,"threads_started":1}
I20260807 15:03:33.243054 13680 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46075
I20260807 15:03:33.243199 13680 heartbeater.cc:464] Registering TS with master...
I20260807 15:03:33.243444 13680 heartbeater.cc:511] Master 127.1.127.254:46075 requested a full tablet report, sending...
I20260807 15:03:33.244029 13314 ts_manager.cc:194] Registered new tserver with Master: ee4ef6c0e984446cb6217c10be8520cc (127.1.127.194:39963)
I20260807 15:03:33.244658 13314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:41081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:03:33.283248 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:59365/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:40225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:03:33.458900 13686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:03:33.459388 13686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:03:33.459534 13686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:03:33.462142 13686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:03:33.462289 13686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:03:33.462385 13686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:03:33.465605 13686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.127.212:59365/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:03:33.466284 13686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:03:33.466805 13686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:03:33.478559 13695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:33.478842 13686 server_base.cc:1034] running on GCE node
W20260807 15:03:33.478590 13693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:03:33.479151 13692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:03:33.479548 13686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:03:33.479923 13686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:03:33.481127 13686 hybrid_clock.cc:648] HybridClock initialized: now 1786115013481099 us; error 44 us; skew 500 ppm
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:03:33.486032 13686 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:03:33.487093 13686 webserver.cc:460] Webserver started at http://127.1.127.195:39079/ using document root <none> and password file <none>
I20260807 15:03:33.487977 13686 fs_manager.cc:359] Metadata directory not provided
I20260807 15:03:33.488183 13686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:03:33.488477 13686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:03:33.521405 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "3a69474b6fde4956bca947c400868bd3"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "5809854d2f8cbb296d54eb72fe444567"
server_key_iv: "ea6e4af773181546c13c31bd2f0246a9"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.522145 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "3a69474b6fde4956bca947c400868bd3"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "5809854d2f8cbb296d54eb72fe444567"
server_key_iv: "ea6e4af773181546c13c31bd2f0246a9"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.529465 13686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 15:03:33.550711 13702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.551568 13686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:03:33.551707 13686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "3a69474b6fde4956bca947c400868bd3"
format_stamp: "Formatted at 2026-08-07 15:03:33 on dist-test-slave-8tdl"
server_key: "5809854d2f8cbb296d54eb72fe444567"
server_key_iv: "ea6e4af773181546c13c31bd2f0246a9"
server_key_version: "kuduclusterkey@0"
I20260807 15:03:33.551831 13686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:03:33.598905 13686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:03:33.602191 13686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:03:33.602527 13686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:03:33.603010 13686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:03:33.603595 13686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:03:33.603706 13686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.603808 13686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:03:33.603874 13686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:03:33.622246 13686 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:37231
I20260807 15:03:33.623175 13686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:03:33.625932 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 13686
I20260807 15:03:33.626106 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:03:33.626401 1535 external_mini_cluster.cc:1428] Setting key 7223af6705a69103477ec158d46e6f4d
I20260807 15:03:33.628682 13822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:37231 every 8 connection(s)
Aug 07 15:03:33 dist-test-slave-8tdl krb5kdc[12816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115013, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:03:33.644667 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.629262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:45033 (local address 127.1.127.254:46075)
0807 15:03:33.629329 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.629331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.629815 (+ 484us) server_negotiation.cc:407] Connection header received
0807 15:03:33.631277 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.631279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.631291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.631350 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:03:33.632800 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.633549 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.636036 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.636300 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.636403 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.640529 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:03:33.640531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:03:33.640531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:03:33.640541 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:03:33.642752 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.643384 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.643386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.643386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.643461 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.643788 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.643789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.643789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.643941 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:03:33.644008 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.644447 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.644565 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:03:33.645051 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.629223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46075 (local address 127.1.127.195:45033)
0807 15:03:33.629612 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.629689 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.631133 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.631469 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.631474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.632099 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:03:33.632646 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.632655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.633695 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.633696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.635854 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.635859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.636449 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.636451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.636692 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.637650 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:03:33.637662 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:03:33.640409 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:03:33.642859 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:33.642872 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:33.642876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:33.643240 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:33.643540 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:03:33.643541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:03:33.643541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:03:33.643706 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:03:33.644014 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:03:33.644022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:03:33.644268 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.644396 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.644629 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
I20260807 15:03:33.646113 13823 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46075
I20260807 15:03:33.646266 13823 heartbeater.cc:464] Registering TS with master...
I20260807 15:03:33.646538 13823 heartbeater.cc:511] Master 127.1.127.254:46075 requested a full tablet report, sending...
I20260807 15:03:33.647119 13314 ts_manager.cc:194] Registered new tserver with Master: 3a69474b6fde4956bca947c400868bd3 (127.1.127.195:37231)
I20260807 15:03:33.647754 13314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:45033
I20260807 15:03:33.650002 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:03:33.663575 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.652562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.1.127.254:46075)
0807 15:03:33.652644 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.652646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.653124 (+ 478us) server_negotiation.cc:407] Connection header received
0807 15:03:33.653182 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.653183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.653196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.653264 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:03:33.654414 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.655313 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.656749 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.657086 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.657196 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.659276 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:03:33.659278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:03:33.659278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:03:33.659299 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:03:33.661679 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.662214 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.662216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.662217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.662312 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:03:33.662652 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:03:33.662654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:03:33.662654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:03:33.662991 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:03:33.663135 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.663295 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.663434 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:03:33.666653 13314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:03:33.669399 13314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:03:33.680786 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.673855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.1.127.193:34703)
0807 15:03:33.674039 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.674045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.675205 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 15:03:33.675323 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.675326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.675346 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.675480 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:03:33.675673 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.677252 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.679348 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.680232 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.680288 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.680353 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.680651 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.680699 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260807 15:03:33.681378 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.673630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34703 (local address 127.0.0.1:35472)
0807 15:03:33.675043 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.675096 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.675180 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.675491 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.675494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.675497 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:03:33.675587 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.675589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.677383 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.677385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.678871 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.678876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.680422 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.680422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.680523 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.680536 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.681215 (+ 679us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.681301 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"thread_start_us":594,"threads_started":1}
I20260807 15:03:33.683367 13458 tablet_service.cc:1467] Processing CreateTablet for tablet c7f9013348324b70ad857204779392ea (DEFAULT_TABLE table=test-table [id=6757dcb70d674ef88aa93413d30e2936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:03:33.683828 13458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f9013348324b70ad857204779392ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:03:33.684785 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.674408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:37231 (local address 127.0.0.1:57394)
0807 15:03:33.678891 (+ 4483us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.678904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.678957 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.679414 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.679415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.679417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:03:33.679539 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.679542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.681841 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.681841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.683214 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.683218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.684553 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.684554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.684619 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.684625 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.684655 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.684700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4441,"thread_start_us":72,"threads_started":1}
I20260807 15:03:33.685181 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.674641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.1.127.195:37231)
0807 15:03:33.679175 (+ 4534us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.679182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.679192 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:03:33.679234 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.679236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.679299 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.679652 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:03:33.679838 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.681715 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.683496 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.684439 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.684909 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.685022 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.685053 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.685097 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4468,"thread_start_us":182,"threads_started":1}
I20260807 15:03:33.687649 13838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.673647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39963 (local address 127.0.0.1:46464)
0807 15:03:33.678855 (+ 5208us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.678871 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.678965 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.681698 (+ 2733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.681699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.681703 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:03:33.681755 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.681757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.683610 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.683612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.684869 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.684872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.687387 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.687389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.687458 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.687463 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.687516 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.687565 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5141,"thread_start_us":1015,"threads_started":1}
I20260807 15:03:33.687563 13748 tablet_service.cc:1467] Processing CreateTablet for tablet c7f9013348324b70ad857204779392ea (DEFAULT_TABLE table=test-table [id=6757dcb70d674ef88aa93413d30e2936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:03:33.687961 13748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f9013348324b70ad857204779392ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:03:33.688395 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.673908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.1.127.194:39963)
0807 15:03:33.674218 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.674222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.679043 (+ 4821us) server_negotiation.cc:407] Connection header received
0807 15:03:33.679157 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.679161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.679182 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.679270 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:03:33.681864 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.683493 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.685003 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.687197 (+ 2194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.687271 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.687347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.688232 (+ 885us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.688285 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":201,"threads_started":1}
I20260807 15:03:33.689821 13842 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: Bootstrap starting.
I20260807 15:03:33.692337 13842 tablet_bootstrap.cc:654] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: Neither blocks nor log segments found. Creating new log.
I20260807 15:03:33.693375 13842 log.cc:824] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: Log is configured to *not* fsync() on all Append() calls
I20260807 15:03:33.694705 13842 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: No bootstrap required, opened a new log
I20260807 15:03:33.694782 13842 ts_tablet_manager.cc:1397] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 15:03:33.697023 13611 tablet_service.cc:1467] Processing CreateTablet for tablet c7f9013348324b70ad857204779392ea (DEFAULT_TABLE table=test-table [id=6757dcb70d674ef88aa93413d30e2936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:03:33.697427 13611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f9013348324b70ad857204779392ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:03:33.700251 13842 raft_consensus.cc:348] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.700471 13842 raft_consensus.cc:374] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:03:33.700506 13842 raft_consensus.cc:729] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b332a07eeba448df96ad9c1f1d08a563, State: Initialized, Role: FOLLOWER
I20260807 15:03:33.700704 13842 consensus_queue.cc:260] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.701237 13842 ts_tablet_manager.cc:1428] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:03:33.701269 13538 heartbeater.cc:503] Master 127.1.127.254:46075 was elected leader, sending a full tablet report...
I20260807 15:03:33.702054 13847 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Bootstrap starting.
I20260807 15:03:33.705348 13847 tablet_bootstrap.cc:654] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Neither blocks nor log segments found. Creating new log.
I20260807 15:03:33.706408 13845 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Bootstrap starting.
I20260807 15:03:33.707672 13847 log.cc:824] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:03:33.710261 13845 tablet_bootstrap.cc:654] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Neither blocks nor log segments found. Creating new log.
I20260807 15:03:33.711571 13845 log.cc:824] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:03:33.712096 13847 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: No bootstrap required, opened a new log
I20260807 15:03:33.712224 13847 ts_tablet_manager.cc:1397] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 15:03:33.712733 13845 tablet_bootstrap.cc:492] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: No bootstrap required, opened a new log
I20260807 15:03:33.712801 13845 ts_tablet_manager.cc:1397] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 15:03:33.717059 13845 raft_consensus.cc:348] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.717290 13845 raft_consensus.cc:374] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:03:33.717326 13845 raft_consensus.cc:729] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a69474b6fde4956bca947c400868bd3, State: Initialized, Role: FOLLOWER
I20260807 15:03:33.717478 13845 consensus_queue.cc:260] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.717967 13845 ts_tablet_manager.cc:1428] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260807 15:03:33.718230 13823 heartbeater.cc:503] Master 127.1.127.254:46075 was elected leader, sending a full tablet report...
I20260807 15:03:33.719902 13847 raft_consensus.cc:348] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.720149 13847 raft_consensus.cc:374] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:03:33.720221 13847 raft_consensus.cc:729] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4ef6c0e984446cb6217c10be8520cc, State: Initialized, Role: FOLLOWER
I20260807 15:03:33.720386 13847 consensus_queue.cc:260] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.724383 13847 ts_tablet_manager.cc:1428] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260807 15:03:33.724702 13680 heartbeater.cc:503] Master 127.1.127.254:46075 was elected leader, sending a full tablet report...
I20260807 15:03:33.738415 13846 raft_consensus.cc:482] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:03:33.738850 13846 raft_consensus.cc:504] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.739689 13846 leader_election.cc:288] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee4ef6c0e984446cb6217c10be8520cc (127.1.127.194:39963), 3a69474b6fde4956bca947c400868bd3 (127.1.127.195:37231)
I20260807 15:03:33.745635 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.740456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:47913 (local address 127.1.127.195:37231)
0807 15:03:33.740583 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.740585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.741718 (+ 1133us) server_negotiation.cc:407] Connection header received
0807 15:03:33.741820 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.741821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.741831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.741863 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:03:33.742278 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.743052 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.744655 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.745203 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.745257 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.745266 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.745500 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.745548 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 15:03:33.745834 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.739887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39963 (local address 127.1.127.193:41575)
0807 15:03:33.740305 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.740695 (+ 390us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.740865 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.741058 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.741059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.741061 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:03:33.741370 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.741373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.743103 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.743104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.744462 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.744467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.745379 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.745381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.745580 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.745586 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.745650 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.745700 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":286,"threads_started":1}
I20260807 15:03:33.745962 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.740338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:37231 (local address 127.1.127.193:47913)
0807 15:03:33.741649 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.741662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:03:33.741746 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:03:33.741928 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:03:33.741929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:03:33.741931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:03:33.742190 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.742192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.743278 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.743279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.744527 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:03:33.744531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.745331 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.745332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:03:33.745416 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.745420 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:03:33.745813 (+ 393us) client_negotiation.cc:240] Negotiation successful
0807 15:03:33.745860 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"mutex_wait_us":24,"thread_start_us":184,"threads_started":1}
I20260807 15:03:33.746289 13759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f9013348324b70ad857204779392ea" candidate_uuid: "b332a07eeba448df96ad9c1f1d08a563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a69474b6fde4956bca947c400868bd3" is_pre_election: true
I20260807 15:03:33.746493 13759 raft_consensus.cc:2454] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b332a07eeba448df96ad9c1f1d08a563 in term 0.
I20260807 15:03:33.746762 13424 leader_election.cc:302] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a69474b6fde4956bca947c400868bd3, b332a07eeba448df96ad9c1f1d08a563; no voters:
I20260807 15:03:33.746870 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.739965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:41575 (local address 127.1.127.194:39963)
0807 15:03:33.740060 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.740063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.740827 (+ 764us) server_negotiation.cc:407] Connection header received
0807 15:03:33.740943 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.740945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.740955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.740986 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:03:33.741548 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.742853 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.744592 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.745227 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.745328 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.745338 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.746740 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.746794 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:03:33.747242 13632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f9013348324b70ad857204779392ea" candidate_uuid: "b332a07eeba448df96ad9c1f1d08a563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4ef6c0e984446cb6217c10be8520cc" is_pre_election: true
I20260807 15:03:33.747413 13632 raft_consensus.cc:2454] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b332a07eeba448df96ad9c1f1d08a563 in term 0.
I20260807 15:03:33.748050 13846 raft_consensus.cc:2781] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:03:33.748142 13846 raft_consensus.cc:482] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:03:33.748171 13846 raft_consensus.cc:3037] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:03:33.749492 13846 raft_consensus.cc:504] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.749837 13632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f9013348324b70ad857204779392ea" candidate_uuid: "b332a07eeba448df96ad9c1f1d08a563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4ef6c0e984446cb6217c10be8520cc"
I20260807 15:03:33.749919 13632 raft_consensus.cc:3037] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:03:33.750887 13632 raft_consensus.cc:2454] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b332a07eeba448df96ad9c1f1d08a563 in term 1.
I20260807 15:03:33.751161 13423 leader_election.cc:302] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b332a07eeba448df96ad9c1f1d08a563, ee4ef6c0e984446cb6217c10be8520cc; no voters:
I20260807 15:03:33.751598 13846 leader_election.cc:288] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [CANDIDATE]: Term 1 election: Requested vote from peers ee4ef6c0e984446cb6217c10be8520cc (127.1.127.194:39963), 3a69474b6fde4956bca947c400868bd3 (127.1.127.195:37231)
I20260807 15:03:33.751657 13846 raft_consensus.cc:2781] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:03:33.751696 13846 raft_consensus.cc:686] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [term 1 LEADER]: Becoming Leader. State: Replica: b332a07eeba448df96ad9c1f1d08a563, State: Running, Role: LEADER
I20260807 15:03:33.751812 13846 consensus_queue.cc:237] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } }
I20260807 15:03:33.752305 13759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f9013348324b70ad857204779392ea" candidate_uuid: "b332a07eeba448df96ad9c1f1d08a563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a69474b6fde4956bca947c400868bd3"
I20260807 15:03:33.752403 13759 raft_consensus.cc:3037] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:03:33.753443 13759 raft_consensus.cc:2454] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b332a07eeba448df96ad9c1f1d08a563 in term 1.
I20260807 15:03:33.754643 13314 catalog_manager.cc:5495] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 reported cstate change: term changed from 0 to 1, leader changed from <none> to b332a07eeba448df96ad9c1f1d08a563 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "b332a07eeba448df96ad9c1f1d08a563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b332a07eeba448df96ad9c1f1d08a563" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 } health_report { overall_health: UNKNOWN } } }
I20260807 15:03:33.771345 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.767238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.1.127.193:34703)
0807 15:03:33.767283 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:03:33.767285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:03:33.767293 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:03:33.767352 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:03:33.767353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:03:33.767378 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:03:33.767451 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:03:33.767658 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.768419 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.769750 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:03:33.770104 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:03:33.770176 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:03:33.770486 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:03:33.770661 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:03:33.770772 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:03:33.771076 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:03:33.771105 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:03:33.775578 13759 raft_consensus.cc:1261] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 1 FOLLOWER]: Refusing update from remote peer b332a07eeba448df96ad9c1f1d08a563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:03:33.775624 13632 raft_consensus.cc:1261] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 1 FOLLOWER]: Refusing update from remote peer b332a07eeba448df96ad9c1f1d08a563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:03:33.776026 13854 consensus_queue.cc:1035] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4ef6c0e984446cb6217c10be8520cc" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:03:33.776203 13846 consensus_queue.cc:1035] T c7f9013348324b70ad857204779392ea P b332a07eeba448df96ad9c1f1d08a563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a69474b6fde4956bca947c400868bd3" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 37231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:03:33.784019 13860 mvcc.cc:205] Tried to move back new op lower bound from 7315927096421257216 to 7315927096335745024. Current Snapshot: MvccSnapshot[applied={T|T < 7315927096421257216}]
I20260807 15:03:33.784195 13857 mvcc.cc:205] Tried to move back new op lower bound from 7315927096421257216 to 7315927096335745024. Current Snapshot: MvccSnapshot[applied={T|T < 7315927096421257216}]
I20260807 15:03:33.785877 13859 mvcc.cc:205] Tried to move back new op lower bound from 7315927096421257216 to 7315927096335745024. Current Snapshot: MvccSnapshot[applied={T|T < 7315927096421257216 or (T in {7315927096421257216})}]
I20260807 15:03:33.789281 13313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:03:33.789407 13313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:03:33.790488 13313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eedcab2940fe44ff8544fa62ab8e1920: Sending DeleteTablet for 3 replicas of tablet c7f9013348324b70ad857204779392ea
I20260807 15:03:33.791090 13611 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f9013348324b70ad857204779392ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:03:33 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:46464
I20260807 15:03:33.791793 13458 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f9013348324b70ad857204779392ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:03:33 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:35472
I20260807 15:03:33.791787 13748 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f9013348324b70ad857204779392ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:03:33 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:57394
I20260807 15:03:33.792728 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 13401
I20260807 15:03:33.793871 13869 tablet_replica.cc:330] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: stopping tablet replica
I20260807 15:03:33.794106 13869 raft_consensus.cc:2229] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:03:33.794404 13869 raft_consensus.cc:2258] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:03:33.795324 13869 ts_tablet_manager.cc:1905] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:03:33.797560 13872 tablet_replica.cc:330] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: stopping tablet replica
I20260807 15:03:33.797758 13869 ts_tablet_manager.cc:1918] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:03:33.797911 13869 log.cc:1196] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/c7f9013348324b70ad857204779392ea
I20260807 15:03:33.798237 13869 ts_tablet_manager.cc:1939] T c7f9013348324b70ad857204779392ea P ee4ef6c0e984446cb6217c10be8520cc: Deleting consensus metadata
I20260807 15:03:33.798439 13872 raft_consensus.cc:2229] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:03:33.799057 13300 catalog_manager.cc:4841] TS ee4ef6c0e984446cb6217c10be8520cc (127.1.127.194:39963): tablet c7f9013348324b70ad857204779392ea (table test-table [id=6757dcb70d674ef88aa93413d30e2936]) successfully deleted
I20260807 15:03:33.799382 13872 raft_consensus.cc:2258] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:03:33.800236 13872 ts_tablet_manager.cc:1905] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:03:33.806855 13872 ts_tablet_manager.cc:1918] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:03:33.807000 13872 log.cc:1196] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/c7f9013348324b70ad857204779392ea
I20260807 15:03:33.807704 13872 ts_tablet_manager.cc:1939] T c7f9013348324b70ad857204779392ea P 3a69474b6fde4956bca947c400868bd3: Deleting consensus metadata
I20260807 15:03:33.808318 13301 catalog_manager.cc:4841] TS 3a69474b6fde4956bca947c400868bd3 (127.1.127.195:37231): tablet c7f9013348324b70ad857204779392ea (table test-table [id=6757dcb70d674ef88aa93413d30e2936]) successfully deleted
W20260807 15:03:33.809705 13301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:34703 (error 108)
I20260807 15:03:33.810472 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 13544
I20260807 15:03:33.817770 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 13686
I20260807 15:03:33.824831 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:03:33.824635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34703 (local address 127.0.0.1:35490)
0807 15:03:33.824712 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 15:03:33.824776 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 15:03:33.825420 13301 catalog_manager.cc:4584] TS b332a07eeba448df96ad9c1f1d08a563 (127.1.127.193:34703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7f9013348324b70ad857204779392ea: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34703: connect: Connection refused (error 111)
I20260807 15:03:33.828452 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 13282
I20260807 15:03:33.835541 1535 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:03:34.181468 1535 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:03:34.181598 1535 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:03:34.526911 1535 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:03:34.527 UTC [12848] LOG: received smart shutdown request
2026-08-07 15:03:34.532 UTC [12848] LOG: background worker "logical replication launcher" (PID 12857) exited with exit code 1
2026-08-07 15:03:34.533 UTC [12852] LOG: shutting down
2026-08-07 15:03:34.758 UTC [12848] LOG: database system is shut down
2026-08-07T15:03:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53416 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:03:34 dist-test-slave-8tdl krb5kdc[13883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:03:34 dist-test-slave-8tdl krb5kdc[13883](info): set up 2 sockets
Aug 07 15:03:34 dist-test-slave-8tdl krb5kdc[13883](info): commencing operation
krb5kdc: starting...
W20260807 15:03:35.971899 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:03:36 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:03:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangeradmin_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangeradmin_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangerlookup_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangerlookup_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
I20260807 15:03:36.175516 1535 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbmaIkX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/postgres -l logfile start
2026-08-07 15:03:39.339 UTC [13915] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:03:39.339 UTC [13915] LOG: listening on IPv4 address "127.1.127.212", port 33717
2026-08-07 15:03:39.344 UTC [13915] LOG: listening on Unix socket "/tmp/.s.PGSQL.33717"
2026-08-07 15:03:39.369 UTC [13918] LOG: database system was shut down at 2026-08-07 15:03:37 UTC
2026-08-07 15:03:39.375 UTC [13915] LOG: database system is ready to accept connections
I20260807 15:03:40.391095 1535 mini_postgres.cc:96] Postgres bound to 33717
2026-08-07 15:03:40.398 UTC [13926] FATAL: database "slave" does not exist
127.1.127.212:33717 - accepting connections
I20260807 15:03:40.399190 1535 mini_ranger.cc:161] Starting Ranger...
I20260807 15:03:40.417018 1535 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:03:41.092964 1535 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:03:41.093041 1535 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin
2026-08-07 15:03:41,383 [I] DB FLAVOR :POSTGRES
2026-08-07 15:03:41,383 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:03:41,383 [I] Checking connection..
2026-08-07 15:03:41,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:03:41,676 [I] Checking connection passed.
2026-08-07 15:03:41,676 [I] --------- Verifying version history table ---------
2026-08-07 15:03:41,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:03:41,952 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:03:41,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:03:42,206 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:03:42,206 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:03:42,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:03:42.458 UTC [13992] WARNING: there is no transaction in progress
2026-08-07 15:03:42,466 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:03:42,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:03:42,745 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:03:42,746 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:03:42,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:03:43,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:03:43,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8tdl.c.gcp-upstream.internal','N') ;"
2026-08-07 15:03:43,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:03:43,839 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:03:43,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:03:44,101 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:03:44,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:03:44,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:03:44,701 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:03:44,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:03:45.911 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:45.932 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:45.952 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:45.972 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:45.992 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.023 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.094 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.104 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.122 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.135 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.148 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.166 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.178 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.188 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.200 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:46.211 UTC [14106] WARNING: there is no transaction in progress
2026-08-07 15:03:47,100 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:03:47,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:03:47,366 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:03:47,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:03:47,648 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:03:47,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:03:47,918 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:03:47,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:03:48,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:03:48,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8tdl.c.gcp-upstream.internal';"
2026-08-07 15:03:48,702 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:03:48,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:03:48,973 [I] DB_PATCHES have already been applied
I20260807 15:03:48.980283 1535 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbmaIkX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:03:48.980412 1535 mini_ranger.cc:193] Using host: 127.1.127.212
I20260807 15:03:48.984732 1535 mini_ranger.cc:227] Ranger admin URL: http://127.1.127.212:58883
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:03:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:03:50.190644 1535 mini_ranger.cc:160] Time spent starting Ranger: real 9.791s user 0.000s sys 0.012s
Aug 07, 2026 3:04:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:04:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 3:04:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:04:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:04:15 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:04:15.737432 1535 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.127.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.127.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangerkms_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.127.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/rangerkms_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.127.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.127.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.127.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/HTTP_127.1.127.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:04:15 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115055, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:04:15.837451 1535 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:04:15.850648 1535 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:04:17.031481 1535 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:04:17.031589 1535 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms
I20260807 15:04:17.031610 1535 mini_ranger_kms.cc:227] Using postgres at 127.1.127.212:33717
2026-08-07 15:04:17,248 [I] DB FLAVOR :POSTGRES
2026-08-07 15:04:17,249 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:04:17,249 [I] Checking connection
2026-08-07 15:04:17,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:04:17,537 [I] connection success
2026-08-07 15:04:17,537 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:04:17,537 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:04:17,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:04:17,872 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:04:17,872 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:04:17,872 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:04:17,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.127.212:33717/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:04:18,212 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:04:18.808847 1535 mini_ranger_kms.cc:310] Created kms service
I20260807 15:04:19.053392 1535 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:04:19.110641 1535 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:04:19 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115055, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:19.552008 1535 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:04:19.552137 1535 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbmaIkX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbmaIkX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:04:19.552157 1535 mini_ranger_kms.cc:240] Using host: 127.1.127.212
I20260807 15:04:19.557220 1535 mini_ranger_kms.cc:276] Ranger KMS PID: 14293
I20260807 15:04:19.557345 1535 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.127.212:42807
Aug 07, 2026 3:04:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:04:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:04:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:04:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:04:21 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115061, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:21.924314 1535 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.087s user 0.000s sys 0.200s
I20260807 15:04:21.924423 1535 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:04:21.924471 1535 mini_ranger_kms.cc:401] 127.1.127.212:42807/kms/v1/keys
Aug 07 15:04:22 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115061, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
Aug 07 15:04:22 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115061, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.127.212@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
Aug 07 15:04:25 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:04:25.248104 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35543
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:35543
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:25.342097 14345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:25.342409 14345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:25.342449 14345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:25.343799 14345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:25.343880 14345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:25.343904 14345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:25.343925 14345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:25.343963 14345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:25.345748 14345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.127.212:42807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:35543
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:25.346082 14345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:25.346343 14345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:25.353788 14350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:25.353813 14353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:25.353823 14351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:25.354481 14345 server_base.cc:1034] running on GCE node
I20260807 15:04:25.354825 14345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:25.355201 14345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:25.356524 14345 hybrid_clock.cc:648] HybridClock initialized: now 1786115065356492 us; error 49 us; skew 500 ppm
Aug 07 15:04:25 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115065, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:25.359642 14345 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:25.360256 14345 webserver.cc:460] Webserver started at http://127.1.127.254:35633/ using document root <none> and password file <none>
I20260807 15:04:25.360541 14345 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:25.360613 14345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:25.360792 14345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:04:25 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115065, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:25.421887 14345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "e7d13f9c0f684b4fbc8c4b99def5706d"
format_stamp: "Formatted at 2026-08-07 15:04:25 on dist-test-slave-8tdl"
server_key: "5b0c4ebdf485d1ca7f8862486876403a"
server_key_iv: "79af961037dbb5bd4ef2f3df9584ed20"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:25.422343 14345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "e7d13f9c0f684b4fbc8c4b99def5706d"
format_stamp: "Formatted at 2026-08-07 15:04:25 on dist-test-slave-8tdl"
server_key: "5b0c4ebdf485d1ca7f8862486876403a"
server_key_iv: "79af961037dbb5bd4ef2f3df9584ed20"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:25.424018 14345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:04:25.438201 14360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:25.438477 14345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:25.438596 14345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e7d13f9c0f684b4fbc8c4b99def5706d"
format_stamp: "Formatted at 2026-08-07 15:04:25 on dist-test-slave-8tdl"
server_key: "5b0c4ebdf485d1ca7f8862486876403a"
server_key_iv: "79af961037dbb5bd4ef2f3df9584ed20"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:25.438753 14345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:25.486927 14345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:25.489399 14345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:25.489610 14345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:25.495023 14345 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:35543
I20260807 15:04:25.495057 14415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:35543 every 8 connection(s)
I20260807 15:04:25.495548 14345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:25.496316 14417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:25.497036 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14345
I20260807 15:04:25.497134 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:04:25.497325 1535 external_mini_cluster.cc:1428] Setting key 71266497deaffbe055a24862425c6a10
I20260807 15:04:25.499302 14417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Bootstrap starting.
I20260807 15:04:25.500131 14417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:25.500515 14417 log.cc:824] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:25.501287 14417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: No bootstrap required, opened a new log
I20260807 15:04:25.503657 14417 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:25.503890 14417 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:25.503974 14417 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d13f9c0f684b4fbc8c4b99def5706d, State: Initialized, Role: FOLLOWER
I20260807 15:04:25.504158 14417 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:25.504256 14417 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:25.504333 14417 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:25.504396 14417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:25.505272 14417 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:25.505384 14417 leader_election.cc:302] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d13f9c0f684b4fbc8c4b99def5706d; no voters:
I20260807 15:04:25.505610 14417 leader_election.cc:288] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:04:25.505666 14422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:25.505925 14422 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 LEADER]: Becoming Leader. State: Replica: e7d13f9c0f684b4fbc8c4b99def5706d, State: Running, Role: LEADER
I20260807 15:04:25.506026 14417 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:25.506072 14422 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
Aug 07 15:04:25 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:25.506526 14422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } } }
I20260807 15:04:25.506659 14422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:25.507218 14422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d13f9c0f684b4fbc8c4b99def5706d. Latest consensus state: current_term: 1 leader_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } } }
I20260807 15:04:25.507674 14417 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:04:25.508054 14432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:25.507472 14422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:25.511678 14432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:25.515115 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:25.497933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.1.127.254:35543)
0807 15:04:25.498821 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:25.498831 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:25.498867 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:04:25.499397 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:25.499405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:25.499471 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:25.499630 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:25.500498 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:25.501858 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:25.503067 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:25.503687 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:25.503793 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:25.506898 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:25.506902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:25.506903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:25.506916 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:25.512585 (+ 5669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:25.513175 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:25.513179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:25.513179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:25.513377 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:25.513684 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:25.513685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:25.513686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:25.514251 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:25.514528 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:25.514606 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:04:25.514917 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":78,"threads_started":1}
I20260807 15:04:25.519362 14432 catalog_manager.cc:1347] Generated new cluster ID: 674f91b8a4c54d9698b7bb8d17ed15c9
I20260807 15:04:25.519528 14432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:04:25.537544 14432 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:04:25.538367 14432 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:25.545765 14432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Generated new TSK 0
I20260807 15:04:25.546085 14432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:04:26.370615 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:26.491093 14462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:26.491349 14462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:26.491557 14462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:26.492828 14462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:26.492913 14462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:26.492962 14462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:26.494493 14462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:26.494804 14462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:26.495118 14462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:26.501688 14469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:26.501801 14471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:26.502071 14462 server_base.cc:1034] running on GCE node
W20260807 15:04:26.502262 14468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:26.502547 14462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:26.502871 14462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:26.504042 14462 hybrid_clock.cc:648] HybridClock initialized: now 1786115066504009 us; error 48 us; skew 500 ppm
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:26.507602 14462 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:26.508232 14462 webserver.cc:460] Webserver started at http://127.1.127.193:42939/ using document root <none> and password file <none>
I20260807 15:04:26.508499 14462 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:26.508577 14462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:26.508736 14462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:26.524084 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "88d86c8d87484f0db54df48098e788ff"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "77e74026ec444c6d787f4abf22178fda"
server_key_iv: "a85014079bdddc07815c9a9273c9b056"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.524720 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "88d86c8d87484f0db54df48098e788ff"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "77e74026ec444c6d787f4abf22178fda"
server_key_iv: "a85014079bdddc07815c9a9273c9b056"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.528893 14462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:04:26.543414 14481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.543677 14462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:26.543767 14462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "88d86c8d87484f0db54df48098e788ff"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "77e74026ec444c6d787f4abf22178fda"
server_key_iv: "a85014079bdddc07815c9a9273c9b056"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.543870 14462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:26.574831 14462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:26.577071 14462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:26.577250 14462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:26.577526 14462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:26.577934 14462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:26.577989 14462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.578059 14462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:26.578104 14462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.585090 14462 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:34497
I20260807 15:04:26.585713 14462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:26.589097 14595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:34497 every 8 connection(s)
I20260807 15:04:26.589322 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14462
I20260807 15:04:26.589445 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:04:26.589686 1535 external_mini_cluster.cc:1428] Setting key 5dcd6a0cc66e664752556095083da5f0
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:26.605716 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:26.588199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.193:34863)
0807 15:04:26.589855 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:26.592746 (+ 2891us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:26.593464 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:26.593882 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:26.593888 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:26.594162 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:26.594462 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:26.594469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.595730 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.595731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:26.597242 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:26.597245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.597883 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.597884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:26.598141 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:26.599871 (+ 1730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:26.599882 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:26.601714 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:26.603930 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:26.603938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:26.603941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:26.604273 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:26.604507 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:26.604508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:26.604508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:26.604648 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:26.604968 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:26.604974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:26.605161 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 15:04:26.605277 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:04:26.605465 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"thread_start_us":91,"threads_started":1}
I20260807 15:04:26.606343 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:26.588534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34863 (local address 127.1.127.254:35543)
0807 15:04:26.592644 (+ 4110us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:26.592647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:26.593624 (+ 977us) server_negotiation.cc:407] Connection header received
0807 15:04:26.593657 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:26.593658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:26.593673 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:26.593734 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:26.594667 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.595545 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.597365 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.597675 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.597742 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:26.601846 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:26.601848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:26.601848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:26.601856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:26.603812 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:26.604365 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:26.604367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:26.604367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:26.604436 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:26.604713 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:26.604714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:26.604714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:26.604864 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:26.604938 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:26.606139 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0807 15:04:26.606237 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4080,"thread_start_us":57,"threads_started":1}
I20260807 15:04:26.607213 14596 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:26.607317 14596 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:26.607548 14596 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:26.608237 14378 ts_manager.cc:194] Registered new tserver with Master: 88d86c8d87484f0db54df48098e788ff (127.1.127.193:34497)
I20260807 15:04:26.609162 14378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:34863
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:04:26.638914 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 15:04:26.739907 14604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:26.740180 14604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:26.740226 14604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:26.742372 14604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:26.742548 14604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:26.742626 14604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:26.745177 14604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:26.745659 14604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:26.746071 14604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:26.752532 14609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:26.752550 14610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:26.753096 14612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:26.753736 14604 server_base.cc:1034] running on GCE node
I20260807 15:04:26.753991 14604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:26.754317 14604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:26.755514 14604 hybrid_clock.cc:648] HybridClock initialized: now 1786115066755471 us; error 70 us; skew 500 ppm
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:26.758469 14604 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:26.759078 14604 webserver.cc:460] Webserver started at http://127.1.127.194:39739/ using document root <none> and password file <none>
I20260807 15:04:26.759338 14604 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:26.759409 14604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:26.759582 14604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:26.776885 14604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "8887dc19710a487f8f3266ccc10148f1"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "7dd4e50c75180b549e903422821c34d9"
server_key_iv: "616358eeb857e85af7b1aa3bc9386d97"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.777379 14604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "8887dc19710a487f8f3266ccc10148f1"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "7dd4e50c75180b549e903422821c34d9"
server_key_iv: "616358eeb857e85af7b1aa3bc9386d97"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.781374 14604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260807 15:04:26.795313 14620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.795565 14604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:26.795675 14604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "8887dc19710a487f8f3266ccc10148f1"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "7dd4e50c75180b549e903422821c34d9"
server_key_iv: "616358eeb857e85af7b1aa3bc9386d97"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:26.795791 14604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:26.826998 14604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:26.829699 14604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:26.829958 14604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:26.830303 14604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:26.830816 14604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:26.830924 14604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.831022 14604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:26.831084 14604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:26.837817 14604 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41241
I20260807 15:04:26.838382 14604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:26.841069 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14604
I20260807 15:04:26.841179 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:04:26.841383 1535 external_mini_cluster.cc:1428] Setting key 57fecf265f32217eb4ba1e08a8361ef3
I20260807 15:04:26.843760 14735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41241 every 8 connection(s)
Aug 07 15:04:26 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115066, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:26.856194 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:26.844595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:36125 (local address 127.1.127.254:35543)
0807 15:04:26.844891 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:26.844894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:26.844977 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:04:26.845735 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:26.845737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:26.845754 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:26.845786 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:26.846756 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.847569 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.849143 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.849456 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.849502 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:26.852457 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:26.852459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:26.852459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:26.852468 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:26.854273 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:26.854898 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:26.854899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:26.854900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:26.854974 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:26.855277 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:26.855278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:26.855278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:26.855427 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:26.855482 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:26.855861 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 15:04:26.856070 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260807 15:04:26.857339 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:26.844594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.194:36125)
0807 15:04:26.844816 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:26.844879 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:26.845602 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:26.845998 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:26.846003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:26.846266 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:26.846601 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:26.846607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.847696 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.847697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:26.849032 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:26.849035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:26.849601 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:26.849602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:26.849763 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:26.850433 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:26.850444 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:26.852325 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:26.854379 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:26.854388 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:26.854391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:26.854772 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:26.855053 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:26.855053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:26.855054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:26.855200 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:26.855518 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:26.855524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:26.855702 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 15:04:26.856791 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0807 15:04:26.857064 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260807 15:04:26.858083 14736 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:26.858183 14736 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:26.858404 14736 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:26.858899 14378 ts_manager.cc:194] Registered new tserver with Master: 8887dc19710a487f8f3266ccc10148f1 (127.1.127.194:41241)
I20260807 15:04:26.859413 14378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:36125
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:04:26.898278 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:27.010461 14741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:27.010771 14741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:27.010821 14741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:27.012665 14741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:27.012737 14741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:27.012779 14741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:27.014528 14741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:27.014891 14741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:27.015187 14741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:27.021224 14748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.021543 14747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.021692 14750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.022992 14741 server_base.cc:1034] running on GCE node
I20260807 15:04:27.023262 14741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:27.023576 14741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:27.024741 14741 hybrid_clock.cc:648] HybridClock initialized: now 1786115067024721 us; error 37 us; skew 500 ppm
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:27.027783 14741 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:27.028286 14741 webserver.cc:460] Webserver started at http://127.1.127.195:37741/ using document root <none> and password file <none>
I20260807 15:04:27.028542 14741 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:27.028621 14741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:27.028779 14741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:27.043417 14741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "0113276f0dcf4678b9bf351490ddb620"
format_stamp: "Formatted at 2026-08-07 15:04:27 on dist-test-slave-8tdl"
server_key: "a054ea7d1ff72773829009c8619143d8"
server_key_iv: "125550a4ab1db700590986a3da06feeb"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.043865 14741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "0113276f0dcf4678b9bf351490ddb620"
format_stamp: "Formatted at 2026-08-07 15:04:27 on dist-test-slave-8tdl"
server_key: "a054ea7d1ff72773829009c8619143d8"
server_key_iv: "125550a4ab1db700590986a3da06feeb"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.049580 14741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 15:04:27.063159 14757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.063406 14741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:27.063519 14741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "0113276f0dcf4678b9bf351490ddb620"
format_stamp: "Formatted at 2026-08-07 15:04:27 on dist-test-slave-8tdl"
server_key: "a054ea7d1ff72773829009c8619143d8"
server_key_iv: "125550a4ab1db700590986a3da06feeb"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.063630 14741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:27.090930 14741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:27.093500 14741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:27.093664 14741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:27.093933 14741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:27.094295 14741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:27.094342 14741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.094373 14741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:27.094396 14741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.103649 14741 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:33863
I20260807 15:04:27.103987 14871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:33863 every 8 connection(s)
I20260807 15:04:27.104135 14741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:27.111599 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14741
I20260807 15:04:27.111774 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:04:27.112087 1535 external_mini_cluster.cc:1428] Setting key 8a7ec05735dd0d59a8ba23e24bbb69f2
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:27.119628 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.105367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:46877 (local address 127.1.127.254:35543)
0807 15:04:27.105412 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:27.105414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:27.105468 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:04:27.106450 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:27.106452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:27.106465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:27.106535 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:27.107343 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.108172 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.109947 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.110250 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.110296 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.114414 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:27.114415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:27.114456 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:27.114466 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:27.116785 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.117567 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.117568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.117569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.117643 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.118633 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.118635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.118635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.118897 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:27.118989 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:27.119383 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 15:04:27.119528 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:04:27.119817 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.105053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.195:46877)
0807 15:04:27.105291 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:27.105353 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:27.106201 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:27.106607 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:27.106612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:27.106926 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:27.107221 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.107228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.108321 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.108322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.109826 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.109830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.110413 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.110414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.110601 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.112559 (+ 1958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:27.112571 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:27.114310 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:27.117061 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.117075 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.117078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.117461 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.118237 (+ 776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.118238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.118239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.118453 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.118990 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:27.118999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:27.119201 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 15:04:27.119338 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 15:04:27.119535 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260807 15:04:27.120651 14872 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:27.120802 14872 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.121026 14872 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:27.121508 14378 ts_manager.cc:194] Registered new tserver with Master: 0113276f0dcf4678b9bf351490ddb620 (127.1.127.195:33863)
I20260807 15:04:27.122022 14378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:46877
I20260807 15:04:27.124342 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:04:27.125036 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14462
I20260807 15:04:27.135493 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14604
I20260807 15:04:27.141461 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14741
I20260807 15:04:27.150942 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14345
I20260807 15:04:27.159286 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35543
--webserver_interface=127.1.127.254
--webserver_port=35633
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:35543
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:27.269039 14875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:27.269276 14875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:27.269302 14875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:27.270728 14875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:27.270787 14875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:27.270802 14875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:27.270815 14875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:27.270828 14875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:27.272907 14875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:35543
--ranger_config_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=35633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:27.273334 14875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:27.273638 14875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:27.279424 14883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.279727 14880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.279742 14881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.279953 14875 server_base.cc:1034] running on GCE node
I20260807 15:04:27.280201 14875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:27.280481 14875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:27.281653 14875 hybrid_clock.cc:648] HybridClock initialized: now 1786115067281640 us; error 34 us; skew 500 ppm
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:27.284651 14875 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:27.285192 14875 webserver.cc:460] Webserver started at http://127.1.127.254:35633/ using document root <none> and password file <none>
I20260807 15:04:27.285465 14875 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:27.285530 14875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:27.334120 14875 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:27.335196 14891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.335425 14875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:27.335541 14875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e7d13f9c0f684b4fbc8c4b99def5706d"
format_stamp: "Formatted at 2026-08-07 15:04:25 on dist-test-slave-8tdl"
server_key: "5b0c4ebdf485d1ca7f8862486876403a"
server_key_iv: "79af961037dbb5bd4ef2f3df9584ed20"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.336035 14875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:27.343708 14875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:27.345940 14875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:27.346078 14875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:27.353879 14875 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:35543
I20260807 15:04:27.354032 14945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:35543 every 8 connection(s)
I20260807 15:04:27.354516 14875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:27.356221 14946 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:04:27.356995 14946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Bootstrap starting.
I20260807 15:04:27.357295 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14875
I20260807 15:04:27.357610 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:34497
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:35543
--webserver_port=42939
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:27.370512 14946 log.cc:824] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:27.372634 14946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:04:27.372995 14946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Bootstrap complete.
I20260807 15:04:27.375664 14946 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:27.375957 14946 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d13f9c0f684b4fbc8c4b99def5706d, State: Initialized, Role: FOLLOWER
I20260807 15:04:27.376181 14946 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:27.376281 14946 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:27.376338 14946 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:27.376396 14946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:04:27.377516 14946 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:27.377648 14946 leader_election.cc:302] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d13f9c0f684b4fbc8c4b99def5706d; no voters:
I20260807 15:04:27.377897 14946 leader_election.cc:288] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:04:27.377938 14950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:04:27.378054 14950 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [term 2 LEADER]: Becoming Leader. State: Replica: e7d13f9c0f684b4fbc8c4b99def5706d, State: Running, Role: LEADER
I20260807 15:04:27.378190 14946 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:27.378230 14950 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } }
I20260807 15:04:27.378754 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } } }
I20260807 15:04:27.378727 14951 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d13f9c0f684b4fbc8c4b99def5706d. Latest consensus state: current_term: 2 leader_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d13f9c0f684b4fbc8c4b99def5706d" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35543 } } }
I20260807 15:04:27.378964 14951 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:27.378904 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:27.379379 14956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:27.380059 14956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:27.380205 14956 catalog_manager.cc:1259] Loaded cluster ID: 674f91b8a4c54d9698b7bb8d17ed15c9
I20260807 15:04:27.380246 14956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:04:27.380595 14946 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:04:27.381645 14956 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:27.381860 14956 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Loaded TSK: 0
I20260807 15:04:27.382102 14956 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:04:27.490979 14948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:27.491317 14948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:27.491365 14948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:27.493579 14948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:27.493667 14948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:27.493716 14948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:27.496573 14948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:34497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=42939
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:27.496873 14948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:27.497184 14948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:27.504835 14986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.507391 14983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.507645 14984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.507851 14948 server_base.cc:1034] running on GCE node
I20260807 15:04:27.508112 14948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:27.508455 14948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:27.509676 14948 hybrid_clock.cc:648] HybridClock initialized: now 1786115067509645 us; error 52 us; skew 500 ppm
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:27.513181 14948 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:27.513863 14948 webserver.cc:460] Webserver started at http://127.1.127.193:42939/ using document root <none> and password file <none>
I20260807 15:04:27.514190 14948 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:27.514288 14948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:27.541783 14948 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:04:27.543823 14994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.544044 14948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:04:27.544181 14948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "88d86c8d87484f0db54df48098e788ff"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "77e74026ec444c6d787f4abf22178fda"
server_key_iv: "a85014079bdddc07815c9a9273c9b056"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.544661 14948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:27.560874 14948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:27.563058 14948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:27.563272 14948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:27.563581 14948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:27.563926 14948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:27.563987 14948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.564116 14948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:27.564144 14948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.571839 14948 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:34497
I20260807 15:04:27.571949 15108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:34497 every 8 connection(s)
I20260807 15:04:27.572356 14948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:27.577780 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 14948
I20260807 15:04:27.578171 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:41241
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:35543
--webserver_port=39739
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:27.587178 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.573297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.193:43707)
0807 15:04:27.573677 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:27.573729 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:27.574402 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:27.575093 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:27.575098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:27.575359 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:27.575650 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.575657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.577009 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.577010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.578319 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.578323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.579324 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.579326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.579479 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.580144 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:27.580156 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:27.581987 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:27.584715 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.584726 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.584729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.585097 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.585460 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.585461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.585462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.585625 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.586224 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:27.586233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:27.586429 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 15:04:27.586562 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 15:04:27.586837 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":91,"threads_started":1}
I20260807 15:04:27.587412 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.573429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:43707 (local address 127.1.127.254:35543)
0807 15:04:27.573677 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:27.573698 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:27.573870 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:04:27.574657 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:27.574663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:27.574789 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:27.574919 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:27.575779 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.576888 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.578436 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.579182 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.579795 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.582135 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:27.582141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:27.582141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:27.582157 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:27.584600 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.585206 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.585209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.585210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.585350 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.585712 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.585714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.585714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.586111 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:27.586775 (+ 664us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:27.586864 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:04:27.587032 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260807 15:04:27.588261 15109 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:27.588382 15109 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.588600 15109 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.589706 15109 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:35543 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:04:27.589793 15109 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.589860 15109 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:27.590330 15109 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.590416 15109 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.590864 15109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:04:27.695411 15113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:27.695708 15113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:27.695751 15113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:27.697122 15113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:27.697199 15113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:27.697247 15113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:27.699406 15113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:41241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=39739
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:27.699702 15113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:27.699965 15113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:27.706550 15119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.706905 15121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.709735 15113 server_base.cc:1034] running on GCE node
W20260807 15:04:27.710850 15118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.711164 15113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:27.711510 15113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:27.712718 15113 hybrid_clock.cc:648] HybridClock initialized: now 1786115067712704 us; error 33 us; skew 500 ppm
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:27.715869 15113 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:27.716571 15113 webserver.cc:460] Webserver started at http://127.1.127.194:39739/ using document root <none> and password file <none>
I20260807 15:04:27.716877 15113 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:27.716989 15113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:27.739311 15113 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260807 15:04:27.740401 15129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.740634 15113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:27.740742 15113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "8887dc19710a487f8f3266ccc10148f1"
format_stamp: "Formatted at 2026-08-07 15:04:26 on dist-test-slave-8tdl"
server_key: "7dd4e50c75180b549e903422821c34d9"
server_key_iv: "616358eeb857e85af7b1aa3bc9386d97"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.741200 15113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:27.756585 15113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:27.759148 15113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:27.759320 15113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:27.759618 15113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:27.759986 15113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:27.760051 15113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.760092 15113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:27.760119 15113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.767629 15113 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41241
I20260807 15:04:27.768209 15113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:27.768898 15245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41241 every 8 connection(s)
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:27.777884 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15113
I20260807 15:04:27.778298 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.769528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:55893 (local address 127.1.127.254:35543)
0807 15:04:27.769571 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:27.769573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:27.769584 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:04:27.770010 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:27.770012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:27.770028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:27.770085 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:27.770772 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.771282 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.772317 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.772545 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.772629 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.775003 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:27.775005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:27.775005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:27.775014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:27.776748 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.777199 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.777200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.777201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.777264 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.777527 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.777528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.777529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.777672 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:27.777745 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:27.778109 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:04:27.778205 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:04:27.778301 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.127.212:42807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:33863
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:35543
--webserver_port=37741
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:27.778498 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.769142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.194:55893)
0807 15:04:27.769319 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:27.769364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:27.769897 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:27.770168 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:27.770172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:27.770348 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:27.770630 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.770637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.771391 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.771392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.772219 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.772221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.772655 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.772656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.772787 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.773447 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:27.773457 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:27.774884 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:27.776848 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.776856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.776859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.777115 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.777334 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.777334 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.777335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.777454 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.777746 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:27.777754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:27.777950 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 15:04:27.778087 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 15:04:27.778242 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
I20260807 15:04:27.779065 15246 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:27.779160 15246 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.779348 15246 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.780063 15246 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:35543 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:04:27.780149 15246 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.780226 15246 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:27.780721 15246 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.780813 15246 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.781220 15246 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:04:27.888993 15249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:27.889272 15249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:27.889319 15249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:27.890604 15249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:27.890681 15249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:27.890754 15249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:27.892369 15249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.127.212:42807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:33863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=37741
--tserver_master_addrs=127.1.127.254:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:27.892681 15249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:27.892951 15249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:27.898850 15254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:27.898831 15255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.899302 15249 server_base.cc:1034] running on GCE node
W20260807 15:04:27.899482 15257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:27.899773 15249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:27.900058 15249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:27.901232 15249 hybrid_clock.cc:648] HybridClock initialized: now 1786115067901219 us; error 27 us; skew 500 ppm
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:27.904008 15249 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:27.904567 15249 webserver.cc:460] Webserver started at http://127.1.127.195:37741/ using document root <none> and password file <none>
I20260807 15:04:27.904819 15249 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:27.904915 15249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:27.930135 15249 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260807 15:04:27.931299 15266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.931483 15249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:27.931586 15249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "0113276f0dcf4678b9bf351490ddb620"
format_stamp: "Formatted at 2026-08-07 15:04:27 on dist-test-slave-8tdl"
server_key: "a054ea7d1ff72773829009c8619143d8"
server_key_iv: "125550a4ab1db700590986a3da06feeb"
server_key_version: "kuduclusterkey@0"
I20260807 15:04:27.932118 15249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:27.954663 15249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:27.957126 15249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:27.957319 15249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:27.957595 15249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:27.958009 15249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:27.958068 15249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.958110 15249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:27.958135 15249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:27.965687 15249 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:33863
I20260807 15:04:27.966229 15249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:27.967664 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15249
I20260807 15:04:27.970150 15380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:33863 every 8 connection(s)
Aug 07 15:04:27 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115067, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:27.977111 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.968674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.1.127.254:35543)
0807 15:04:27.968726 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:27.968728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:27.968739 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:04:27.968769 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:27.968771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:27.968781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:27.968848 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:27.969722 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.970905 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.972040 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.972317 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.972390 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.973828 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:27.973830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:27.973831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:27.973838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:27.975456 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.976067 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.976069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.976070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.976135 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.976353 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.976354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.976355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.976525 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:27.976699 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:27.976904 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:04:27.976997 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:04:27.980840 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.968903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:53927 (local address 127.1.127.254:35543)
0807 15:04:27.969069 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:27.969071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:27.969079 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:04:27.969100 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:27.969100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:27.969110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:27.971316 (+ 2206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:27.971348 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.971816 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.973213 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.973426 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.973514 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.976211 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:27.976212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:27.976212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:27.976217 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:27.979098 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.979746 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.979748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.979748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.979822 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:27.980101 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:27.980102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:27.980103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:27.980229 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:27.980316 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:27.980689 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 15:04:27.980761 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
I20260807 15:04:27.981137 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:27.967199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35543 (local address 127.1.127.195:53927)
0807 15:04:27.967446 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:27.967527 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:27.968191 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:27.969409 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:27.969414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:27.969654 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:27.971229 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.971235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.971928 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.971929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.973103 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:27.973105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:27.973547 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:27.973548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:27.973679 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:27.974255 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:27.974264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:27.975957 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:27.979217 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.979227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.979230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.979665 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.979892 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:27.979893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:27.979893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:27.980029 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:27.980339 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:27.980346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:27.980534 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 15:04:27.980681 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 15:04:27.980864 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260807 15:04:27.981655 15381 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35543
I20260807 15:04:27.981765 15381 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.981942 15381 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.982709 15381 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:35543 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:04:27.982789 15381 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.982861 15381 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:27.983172 15381 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:27.983253 15381 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
W20260807 15:04:27.983628 15381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:04:28 dist-test-slave-8tdl krb5kdc[13883](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:28.591075 15109 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:28.591250 15109 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:28.591689 14908 ts_manager.cc:194] Registered new tserver with Master: 88d86c8d87484f0db54df48098e788ff (127.1.127.193:34497)
I20260807 15:04:28.592710 14908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:43707
I20260807 15:04:28.781426 15246 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:28.781617 15246 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:28.782172 14908 ts_manager.cc:194] Registered new tserver with Master: 8887dc19710a487f8f3266ccc10148f1 (127.1.127.194:41241)
I20260807 15:04:28.782764 14908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:55893
Aug 07 15:04:28 dist-test-slave-8tdl krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115068, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for HTTP/127.1.127.212@KRBTEST.COM
I20260807 15:04:28.983821 15381 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:28.983997 15381 heartbeater.cc:511] Master 127.1.127.254:35543 requested a full tablet report, sending...
I20260807 15:04:28.984438 14908 ts_manager.cc:194] Registered new tserver with Master: 0113276f0dcf4678b9bf351490ddb620 (127.1.127.195:33863)
I20260807 15:04:28.984733 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:04:28.984982 14908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:53927
I20260807 15:04:28.996014 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:28.986457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.1.127.254:35543)
0807 15:04:28.986779 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:28.986781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:28.986788 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:04:28.986812 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:28.986813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:28.986823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:28.987591 (+ 768us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:28.987633 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:28.988971 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:28.990420 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:28.990744 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:28.991133 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:28.992404 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:28.992405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:28.992406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:28.992414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:28.994257 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:28.994725 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:28.994726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:28.994727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:28.994789 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:28.995058 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:28.995059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:28.995059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:28.995202 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:28.995765 (+ 563us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:28.995810 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:04:28.995904 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":84,"threads_started":1}
I20260807 15:04:28.998472 14908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:04:29.000953 14908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:04:29.014343 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.009108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41241 (local address 127.0.0.1:37058)
0807 15:04:29.009731 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.009776 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:29.009873 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:29.010096 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:29.010099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:29.010101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:29.010176 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.010178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.011421 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.011422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.012917 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.012921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.013976 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.013977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.014072 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.014086 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:04:29.014154 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:04:29.014218 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":73,"threads_started":1}
I20260807 15:04:29.015146 15413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.009248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.1.127.194:41241)
0807 15:04:29.009468 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.009474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.009858 (+ 384us) server_negotiation.cc:407] Connection header received
0807 15:04:29.009965 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.009968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.009987 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.010063 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:29.010267 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.011333 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.013043 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.013796 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.013872 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.013934 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.014995 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.015056 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260807 15:04:29.017326 15414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.008805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:33863 (local address 127.0.0.1:35436)
0807 15:04:29.009566 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.010723 (+ 1157us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:29.010782 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:29.011069 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:29.011070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:29.011072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:29.011116 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.011118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.014352 (+ 3234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.014353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.015931 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.015934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.017059 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.017061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.017122 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.017127 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:04:29.017159 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 15:04:29.017203 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":136,"threads_started":1}
I20260807 15:04:29.017753 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.008951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.1.127.195:33863)
0807 15:04:29.010041 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.010046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.010846 (+ 800us) server_negotiation.cc:407] Connection header received
0807 15:04:29.010884 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.010886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.010902 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.010971 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:29.011183 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.012305 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.016047 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.016951 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.017487 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.017600 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.017631 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.017675 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":82,"threads_started":1}
I20260807 15:04:29.019074 15300 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6c9dd00614636b85f7a83aed0884c (DEFAULT_TABLE table=test-table [id=91d18b38c72441d9bb7dbd528d00d933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:29.019134 15169 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6c9dd00614636b85f7a83aed0884c (DEFAULT_TABLE table=test-table [id=91d18b38c72441d9bb7dbd528d00d933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:29.019500 15300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6c9dd00614636b85f7a83aed0884c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:29.019541 15169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6c9dd00614636b85f7a83aed0884c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:29.020480 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.009273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34497 (local address 127.0.0.1:42392)
0807 15:04:29.014776 (+ 5503us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.014791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:29.014899 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:29.016444 (+ 1545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:29.016445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:29.016448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:29.016485 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.016487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.018114 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.018115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.019188 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.019191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.020237 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.020238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.020300 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.020303 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:04:29.020354 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:04:29.020393 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5483,"thread_start_us":78,"threads_started":1}
I20260807 15:04:29.020738 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.009492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.1.127.193:34497)
0807 15:04:29.016179 (+ 6687us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.016186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.016197 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:04:29.016235 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.016237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.016257 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.016343 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:29.016566 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.018018 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.019296 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.020070 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.020144 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.020200 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.020594 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.020660 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6642,"thread_start_us":111,"threads_started":1}
I20260807 15:04:29.022753 15416 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Bootstrap starting.
I20260807 15:04:29.024087 15042 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6c9dd00614636b85f7a83aed0884c (DEFAULT_TABLE table=test-table [id=91d18b38c72441d9bb7dbd528d00d933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:29.024497 15042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6c9dd00614636b85f7a83aed0884c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:29.025668 15417 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Bootstrap starting.
I20260807 15:04:29.026185 15416 tablet_bootstrap.cc:654] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:29.026979 15416 log.cc:824] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:29.027496 15417 tablet_bootstrap.cc:654] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:29.028085 15417 log.cc:824] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:29.028283 15416 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: No bootstrap required, opened a new log
I20260807 15:04:29.028350 15416 ts_tablet_manager.cc:1397] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260807 15:04:29.030567 15416 raft_consensus.cc:348] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.030771 15416 raft_consensus.cc:374] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:29.030817 15416 raft_consensus.cc:729] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0113276f0dcf4678b9bf351490ddb620, State: Initialized, Role: FOLLOWER
I20260807 15:04:29.030879 15419 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Bootstrap starting.
I20260807 15:04:29.031137 15416 consensus_queue.cc:260] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.031648 15419 tablet_bootstrap.cc:654] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:29.031661 15416 ts_tablet_manager.cc:1428] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 15:04:29.031776 15381 heartbeater.cc:503] Master 127.1.127.254:35543 was elected leader, sending a full tablet report...
I20260807 15:04:29.032002 15419 log.cc:824] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:29.032994 15417 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: No bootstrap required, opened a new log
I20260807 15:04:29.033074 15417 ts_tablet_manager.cc:1397] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 15:04:29.044523 15417 raft_consensus.cc:348] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.044775 15417 raft_consensus.cc:374] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:29.044813 15417 raft_consensus.cc:729] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8887dc19710a487f8f3266ccc10148f1, State: Initialized, Role: FOLLOWER
I20260807 15:04:29.044966 15417 consensus_queue.cc:260] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.045320 15417 ts_tablet_manager.cc:1428] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Time spent starting tablet: real 0.012s user 0.000s sys 0.007s
I20260807 15:04:29.046005 15419 tablet_bootstrap.cc:492] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: No bootstrap required, opened a new log
I20260807 15:04:29.046110 15419 ts_tablet_manager.cc:1397] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260807 15:04:29.050932 15246 heartbeater.cc:503] Master 127.1.127.254:35543 was elected leader, sending a full tablet report...
I20260807 15:04:29.051635 15419 raft_consensus.cc:348] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.051846 15419 raft_consensus.cc:374] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:29.051882 15419 raft_consensus.cc:729] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d86c8d87484f0db54df48098e788ff, State: Initialized, Role: FOLLOWER
I20260807 15:04:29.052033 15419 consensus_queue.cc:260] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.052423 15419 ts_tablet_manager.cc:1428] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 15:04:29.052559 15109 heartbeater.cc:503] Master 127.1.127.254:35543 was elected leader, sending a full tablet report...
W20260807 15:04:29.074476 15110 tablet.cc:2369] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:29.130455 15421 raft_consensus.cc:482] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:29.130640 15421 raft_consensus.cc:504] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.131088 15421 leader_election.cc:288] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8887dc19710a487f8f3266ccc10148f1 (127.1.127.194:41241), 88d86c8d87484f0db54df48098e788ff (127.1.127.193:34497)
I20260807 15:04:29.135702 15413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.131560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:48757 (local address 127.1.127.194:41241)
0807 15:04:29.131707 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.131709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.131949 (+ 240us) server_negotiation.cc:407] Connection header received
0807 15:04:29.132061 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.132063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.132073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.132103 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:29.132330 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.133083 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.134635 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.135242 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.135298 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.135308 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.135573 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.135622 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 15:04:29.136221 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.131466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41241 (local address 127.1.127.195:48757)
0807 15:04:29.131857 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.131871 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:29.131979 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:29.132175 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:29.132175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:29.132177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:29.132258 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.132260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.133188 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.133189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.134508 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.134512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.135402 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.135403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.135470 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.135474 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:04:29.136069 (+ 595us) client_negotiation.cc:240] Negotiation successful
0807 15:04:29.136120 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":316,"threads_started":1}
I20260807 15:04:29.136916 15198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6c9dd00614636b85f7a83aed0884c" candidate_uuid: "0113276f0dcf4678b9bf351490ddb620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8887dc19710a487f8f3266ccc10148f1" is_pre_election: true
I20260807 15:04:29.137116 15198 raft_consensus.cc:2454] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0113276f0dcf4678b9bf351490ddb620 in term 0.
I20260807 15:04:29.137521 15267 leader_election.cc:302] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0113276f0dcf4678b9bf351490ddb620, 8887dc19710a487f8f3266ccc10148f1; no voters:
I20260807 15:04:29.137672 15421 raft_consensus.cc:2781] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:04:29.137713 15421 raft_consensus.cc:482] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:04:29.137735 15421 raft_consensus.cc:3037] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:29.138442 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.132315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:56677 (local address 127.1.127.193:34497)
0807 15:04:29.132384 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.132386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.132646 (+ 260us) server_negotiation.cc:407] Connection header received
0807 15:04:29.132827 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.132828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.132849 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.132896 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:29.133446 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.134672 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.137327 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.137893 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.137948 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.137958 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.138322 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.138373 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:04:29.138844 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.131973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34497 (local address 127.1.127.195:56677)
0807 15:04:29.132534 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.132547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:29.132690 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:29.133006 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:29.133007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:29.133018 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:29.133290 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.133292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.134870 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.134871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.136448 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:29.136451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.138155 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.138156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:29.138234 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.138239 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:04:29.138702 (+ 463us) client_negotiation.cc:240] Negotiation successful
0807 15:04:29.138755 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":161,"threads_started":1}
I20260807 15:04:29.139482 15421 raft_consensus.cc:504] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.139485 15062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6c9dd00614636b85f7a83aed0884c" candidate_uuid: "0113276f0dcf4678b9bf351490ddb620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d86c8d87484f0db54df48098e788ff" is_pre_election: true
I20260807 15:04:29.139642 15421 leader_election.cc:288] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [CANDIDATE]: Term 1 election: Requested vote from peers 8887dc19710a487f8f3266ccc10148f1 (127.1.127.194:41241), 88d86c8d87484f0db54df48098e788ff (127.1.127.193:34497)
I20260807 15:04:29.139668 15062 raft_consensus.cc:2454] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0113276f0dcf4678b9bf351490ddb620 in term 0.
I20260807 15:04:29.139999 15062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6c9dd00614636b85f7a83aed0884c" candidate_uuid: "0113276f0dcf4678b9bf351490ddb620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88d86c8d87484f0db54df48098e788ff"
I20260807 15:04:29.140072 15062 raft_consensus.cc:3037] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:29.141371 15062 raft_consensus.cc:2454] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0113276f0dcf4678b9bf351490ddb620 in term 1.
I20260807 15:04:29.141607 15267 leader_election.cc:302] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0113276f0dcf4678b9bf351490ddb620, 88d86c8d87484f0db54df48098e788ff; no voters:
I20260807 15:04:29.142287 15421 raft_consensus.cc:2781] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:29.142467 15421 raft_consensus.cc:686] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 1 LEADER]: Becoming Leader. State: Replica: 0113276f0dcf4678b9bf351490ddb620, State: Running, Role: LEADER
I20260807 15:04:29.142577 15421 consensus_queue.cc:237] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } }
I20260807 15:04:29.143296 15198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6c9dd00614636b85f7a83aed0884c" candidate_uuid: "0113276f0dcf4678b9bf351490ddb620" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8887dc19710a487f8f3266ccc10148f1"
I20260807 15:04:29.143409 15198 raft_consensus.cc:3037] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:29.143585 14908 catalog_manager.cc:5495] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0113276f0dcf4678b9bf351490ddb620 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "0113276f0dcf4678b9bf351490ddb620" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0113276f0dcf4678b9bf351490ddb620" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 33863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 } health_report { overall_health: UNKNOWN } } }
I20260807 15:04:29.145032 15198 raft_consensus.cc:2454] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0113276f0dcf4678b9bf351490ddb620 in term 1.
I20260807 15:04:29.178232 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.174374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.1.127.195:33863)
0807 15:04:29.174419 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:29.174422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:29.174434 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:29.174481 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:29.174482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:29.174506 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:29.174606 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:04:29.174824 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.175651 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.176792 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:29.177224 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:29.177327 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:29.177694 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:04:29.177881 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:04:29.177979 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:29.178089 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:04:29.178119 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:04:29.181206 15198 raft_consensus.cc:1261] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 1 FOLLOWER]: Refusing update from remote peer 0113276f0dcf4678b9bf351490ddb620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:29.181226 15062 raft_consensus.cc:1261] T 65c6c9dd00614636b85f7a83aed0884c P 88d86c8d87484f0db54df48098e788ff [term 1 FOLLOWER]: Refusing update from remote peer 0113276f0dcf4678b9bf351490ddb620: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:29.181746 15427 consensus_queue.cc:1035] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88d86c8d87484f0db54df48098e788ff" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 34497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:29.181892 15421 consensus_queue.cc:1035] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8887dc19710a487f8f3266ccc10148f1" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:29.192564 15431 mvcc.cc:205] Tried to move back new op lower bound from 7315927323362533376 to 7315927323209134080. Current Snapshot: MvccSnapshot[applied={T|T < 7315927323362533376}]
I20260807 15:04:29.192677 14908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:29.192803 14908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:29.192796 15432 mvcc.cc:205] Tried to move back new op lower bound from 7315927323362533376 to 7315927323209134080. Current Snapshot: MvccSnapshot[applied={T|T < 7315927323362533376}]
I20260807 15:04:29.193349 15430 mvcc.cc:205] Tried to move back new op lower bound from 7315927323362533376 to 7315927323209134080. Current Snapshot: MvccSnapshot[applied={T|T < 7315927323362533376 or (T in {7315927323362533376})}]
I20260807 15:04:29.193650 14908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7d13f9c0f684b4fbc8c4b99def5706d: Sending DeleteTablet for 3 replicas of tablet 65c6c9dd00614636b85f7a83aed0884c
I20260807 15:04:29.194275 15169 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6c9dd00614636b85f7a83aed0884c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:29 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:37058
I20260807 15:04:29.194837 15042 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6c9dd00614636b85f7a83aed0884c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:29 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:42392
I20260807 15:04:29.195319 15300 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6c9dd00614636b85f7a83aed0884c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:29 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:35436
I20260807 15:04:29.196022 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14948
I20260807 15:04:29.201629 15440 tablet_replica.cc:330] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: stopping tablet replica
I20260807 15:04:29.201804 15440 raft_consensus.cc:2229] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:29.201951 15440 raft_consensus.cc:2258] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:29.202765 15440 ts_tablet_manager.cc:1905] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:29.205848 15440 ts_tablet_manager.cc:1918] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:29.205974 15440 log.cc:1196] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/65c6c9dd00614636b85f7a83aed0884c
I20260807 15:04:29.206264 15440 ts_tablet_manager.cc:1939] T 65c6c9dd00614636b85f7a83aed0884c P 8887dc19710a487f8f3266ccc10148f1: Deleting consensus metadata
I20260807 15:04:29.206529 15442 tablet_replica.cc:330] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: stopping tablet replica
I20260807 15:04:29.206704 15442 raft_consensus.cc:2229] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:04:29.206904 15442 raft_consensus.cc:2258] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:29.207417 14892 catalog_manager.cc:4841] TS 8887dc19710a487f8f3266ccc10148f1 (127.1.127.194:41241): tablet 65c6c9dd00614636b85f7a83aed0884c (table test-table [id=91d18b38c72441d9bb7dbd528d00d933]) successfully deleted
W20260807 15:04:29.207615 14892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:34497 (error 108)
I20260807 15:04:29.207864 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15113
I20260807 15:04:29.210027 15442 ts_tablet_manager.cc:1905] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:29.212374 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:29.212196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:34497 (local address 127.0.0.1:42406)
0807 15:04:29.212246 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:29.212308 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:04:29.212356 15442 ts_tablet_manager.cc:1918] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 15:04:29.212518 14892 catalog_manager.cc:4584] TS 88d86c8d87484f0db54df48098e788ff (127.1.127.193:34497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65c6c9dd00614636b85f7a83aed0884c: Network error: Client connection negotiation failed: client connection to 127.1.127.193:34497: connect: Connection refused (error 111)
I20260807 15:04:29.212651 15442 log.cc:1196] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/65c6c9dd00614636b85f7a83aed0884c
I20260807 15:04:29.212986 15442 ts_tablet_manager.cc:1939] T 65c6c9dd00614636b85f7a83aed0884c P 0113276f0dcf4678b9bf351490ddb620: Deleting consensus metadata
I20260807 15:04:29.213500 14892 catalog_manager.cc:4841] TS 0113276f0dcf4678b9bf351490ddb620 (127.1.127.195:33863): tablet 65c6c9dd00614636b85f7a83aed0884c (table test-table [id=91d18b38c72441d9bb7dbd528d00d933]) successfully deleted
I20260807 15:04:29.217298 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15249
I20260807 15:04:29.225256 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 14875
I20260807 15:04:29.236469 1535 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:04:29.582235 1535 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:04:29.582466 1535 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:04:29.927788 1535 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:04:29.927 UTC [13915] LOG: received smart shutdown request
2026-08-07 15:04:29.933 UTC [13915] LOG: background worker "logical replication launcher" (PID 13924) exited with exit code 1
2026-08-07 15:04:29.934 UTC [13919] LOG: shutting down
2026-08-07 15:04:30.151 UTC [13915] LOG: database system is shut down
2026-08-07T15:04:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55418 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:04:30 dist-test-slave-8tdl krb5kdc[15452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:04:30 dist-test-slave-8tdl krb5kdc[15452](info): set up 2 sockets
Aug 07 15:04:30 dist-test-slave-8tdl krb5kdc[15452](info): commencing operation
krb5kdc: starting...
W20260807 15:04:31.414206 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:04:31.559083 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:45227
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:31.645435 15466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:31.645660 15466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:31.645685 15466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:31.646970 15466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:31.647034 15466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:31.647050 15466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:31.647063 15466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:31.647076 15466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:31.648967 15466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:45227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:31.649305 15466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:31.649598 15466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:31.655150 15472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:31.655166 15474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:31.655149 15466 server_base.cc:1034] running on GCE node
W20260807 15:04:31.655144 15471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:31.655635 15466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:31.655953 15466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:31.657135 15466 hybrid_clock.cc:648] HybridClock initialized: now 1786115071657111 us; error 42 us; skew 500 ppm
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:31.659991 15466 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:31.660526 15466 webserver.cc:460] Webserver started at http://127.1.127.254:41569/ using document root <none> and password file <none>
I20260807 15:04:31.660784 15466 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:31.660856 15466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:31.661013 15466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:31.662197 15466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "27aaff21421f4529b3e0ffde21844c59"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "4db80980c061c76f70c6143d6a4f47ea"
server_key_iv: "3dabd17bf2e0fb676e20d99e58758388"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.662586 15466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "27aaff21421f4529b3e0ffde21844c59"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "4db80980c061c76f70c6143d6a4f47ea"
server_key_iv: "3dabd17bf2e0fb676e20d99e58758388"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.664184 15466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:31.665073 15481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:31.665288 15466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:31.665400 15466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "27aaff21421f4529b3e0ffde21844c59"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "4db80980c061c76f70c6143d6a4f47ea"
server_key_iv: "3dabd17bf2e0fb676e20d99e58758388"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.665494 15466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:31.677076 15466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:31.679566 15466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:31.679759 15466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:31.684520 15466 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:45227
I20260807 15:04:31.684561 15534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:45227 every 8 connection(s)
I20260807 15:04:31.684942 15466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:31.685304 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15466
I20260807 15:04:31.685457 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:04:31.685724 1535 external_mini_cluster.cc:1428] Setting key 679223aaea4bed455aec3e1740656dc0
I20260807 15:04:31.686170 15535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:31.688584 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.127.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.253@KRBTEST.COM" created.
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:31.705134 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.689466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45227 (local address 127.0.0.1:44428)
0807 15:04:31.689683 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.689734 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:31.690336 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:31.690755 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:31.690759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:31.691182 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:31.691431 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.691438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.692605 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.692606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.693981 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.693985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.694584 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.694585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.694716 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.695361 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:31.695373 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:31.700652 (+ 5279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:31.703283 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.703292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.703294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.703570 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.703872 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.703873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.703873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.704046 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.704477 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:31.704483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:31.704557 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 15:04:31.704662 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 15:04:31.704936 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":101,"thread_start_us":109,"threads_started":1}
I20260807 15:04:31.705137 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.689558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.1.127.254:45227)
0807 15:04:31.689769 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:31.689774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:31.689866 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:04:31.690538 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:31.690540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:31.690559 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:31.690670 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:31.691577 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.692463 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.694110 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.694471 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.694576 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.700780 (+ 6204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:31.700794 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:31.700795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:31.700805 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:31.703179 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.703693 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.703696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.703696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.703791 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.704123 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.704124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.704125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.704380 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:31.704500 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:31.704691 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:04:31.704854 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260807 15:04:31.706126 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:31.706647 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.706447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46578)
0807 15:04:31.706541 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.706607 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 15:04:31.706873 15485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
I20260807 15:04:31.707518 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.707412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46588)
0807 15:04:31.707464 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.707495 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:04:31.707659 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:39837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
I20260807 15:04:31.722663 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:39837
--webserver_interface=127.1.127.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:31.756968 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } attempt: 1
I20260807 15:04:31.757622 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.757414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46594)
0807 15:04:31.757532 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.757573 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 15:04:31.757974 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.757890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46602)
0807 15:04:31.757925 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.757951 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 15:04:31.758057 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:39837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
W20260807 15:04:31.815572 15542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:31.815856 15542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:31.815897 15542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:31.817122 15542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:31.817206 15542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:31.817230 15542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:31.817251 15542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:31.817279 15542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:31.819020 15542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.127.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:31.819331 15542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:31.819582 15542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:31.824863 15548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:31.824862 15550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:31.824862 15547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:31.825260 15542 server_base.cc:1034] running on GCE node
I20260807 15:04:31.825562 15542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:31.825836 15542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:31.826192 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } attempt: 2
I20260807 15:04:31.826574 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.826460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46618)
0807 15:04:31.826515 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.826546 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:04:31.826903 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.826820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46626)
0807 15:04:31.826865 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.826888 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 15:04:31.827008 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:39837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:39837: connect: Connection refused (error 111)
I20260807 15:04:31.827041 15542 hybrid_clock.cc:648] HybridClock initialized: now 1786115071827027 us; error 32 us; skew 500 ppm
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:31.829855 15542 init.cc:377] Logged in from keytab as kudu/127.1.127.253@KRBTEST.COM (short username kudu)
I20260807 15:04:31.830415 15542 webserver.cc:460] Webserver started at http://127.1.127.253:35323/ using document root <none> and password file <none>
I20260807 15:04:31.830653 15542 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:31.830763 15542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:31.830929 15542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:31.832058 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data/instance:
uuid: "814a776c366c4e49a7e14142fb929ca7"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "e8bc42cdf82580e5744e78d10353902d"
server_key_iv: "3fbced2d6021060a3d9f62a5b28bdb1f"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.832376 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal/instance:
uuid: "814a776c366c4e49a7e14142fb929ca7"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "e8bc42cdf82580e5744e78d10353902d"
server_key_iv: "3fbced2d6021060a3d9f62a5b28bdb1f"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.833766 15542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:04:31.834611 15557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:31.834864 15542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:31.835021 15542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "814a776c366c4e49a7e14142fb929ca7"
format_stamp: "Formatted at 2026-08-07 15:04:31 on dist-test-slave-8tdl"
server_key: "e8bc42cdf82580e5744e78d10353902d"
server_key_iv: "3fbced2d6021060a3d9f62a5b28bdb1f"
server_key_version: "encryptionkey@0"
I20260807 15:04:31.835155 15542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:31.873191 15542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:31.875494 15542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:31.875700 15542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:31.880223 15542 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.253:39837
I20260807 15:04:31.880264 15611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.253:39837 every 8 connection(s)
I20260807 15:04:31.880671 15542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
I20260807 15:04:31.881163 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15542
I20260807 15:04:31.881301 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-1/wal/instance
I20260807 15:04:31.881528 1535 external_mini_cluster.cc:1428] Setting key c29668e7d20faacf5e6452fb2979ba07
I20260807 15:04:31.881872 15612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:31.884254 15612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.127.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.252@KRBTEST.COM" created.
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:31.899852 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.885100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45227 (local address 127.0.0.1:44434)
0807 15:04:31.885328 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.885391 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:31.885965 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:31.886360 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:31.886364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:31.886657 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:31.886964 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.886971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.888103 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.888104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.889199 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.889202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.889933 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.889935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.890072 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.890816 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:31.890911 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:31.895832 (+ 4921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:31.898101 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.898114 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.898117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.898417 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.898665 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.898665 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.898666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.898865 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.899264 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:31.899272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:31.899416 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:04:31.899513 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:04:31.899665 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260807 15:04:31.899854 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.885218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.1.127.254:45227)
0807 15:04:31.885281 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:31.885284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:31.885497 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:04:31.886133 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:31.886136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:31.886151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:31.886234 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:31.887089 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.887943 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.889320 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.889683 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.889776 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.895995 (+ 6219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:31.896006 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:31.896006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:31.896017 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:31.897943 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.898507 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.898509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.898510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.898586 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.898940 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.898941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.898942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.899179 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:31.899257 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:31.899587 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 15:04:31.899703 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:04:31.900707 15612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:04:31.911089 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.901021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46632)
0807 15:04:31.901087 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.901099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:31.901145 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:31.901396 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:31.901397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:31.901556 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:31.901607 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.901609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.902524 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.902525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.903108 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.903110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.903598 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.903599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.903673 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.904217 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:31.904221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:31.907741 (+ 3520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:31.909814 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.909815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.909815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.909992 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.910219 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.910220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.910220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.910279 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.910588 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:31.910589 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:31.910629 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 15:04:31.910834 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 15:04:31.910972 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:04:31.911309 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.901078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46632 (local address 127.1.127.253:39837)
0807 15:04:31.901263 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:31.901267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:31.901285 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:04:31.901301 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:31.901302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:31.901313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:31.901396 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:31.901682 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.902405 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.903192 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.903517 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.903603 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.907938 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:31.907951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:31.907952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:31.907959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:31.909714 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.910084 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.910087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.910088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.910150 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.910328 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.910329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.910329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.910504 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:31.910602 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:31.910886 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 15:04:31.911161 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260807 15:04:31.911924 15612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:31.912386 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.912235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59684)
0807 15:04:31.912295 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.912357 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 15:04:31.912667 15559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
I20260807 15:04:31.913228 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.913105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59692)
0807 15:04:31.913174 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.913205 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:04:31.913491 15612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
I20260807 15:04:31.916255 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:34411
--webserver_interface=127.1.127.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:31.918181 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } attempt: 3
Aug 07 15:04:31 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:04:31.925661 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.918574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46638)
0807 15:04:31.918645 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.918665 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:31.918771 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:31.919006 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:31.919007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:31.919177 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:31.919239 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.919241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.919936 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.919938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.920542 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:31.920544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.920952 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.920953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:31.921038 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.921510 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:31.921522 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:31.922796 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:31.924523 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.924524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.924525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.924713 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.924951 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:31.924952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:31.924953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:31.925021 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:31.925300 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:31.925302 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:31.925352 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:04:31.925443 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:04:31.925555 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:04:31.925661 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.918658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.1.127.253:39837)
0807 15:04:31.918761 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:31.918762 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:31.918774 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:31.918841 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:31.918842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:31.918854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:31.918999 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:31.919323 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.919839 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.920631 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:31.920855 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:31.920942 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:31.922895 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:31.922896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:31.922896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:31.922907 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:31.924412 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.924805 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.924806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.924807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.924869 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:31.925096 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:31.925097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:31.925097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:31.925221 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:31.925297 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:31.925471 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:04:31.925558 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:04:31.926275 15535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:31.926733 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.926573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59694)
0807 15:04:31.926624 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.926659 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:04:31.927253 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.927159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59698)
0807 15:04:31.927205 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.927229 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:04:31.927413 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
I20260807 15:04:31.958606 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 1
I20260807 15:04:31.959280 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.959120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59702)
0807 15:04:31.959189 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.959241 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:04:31.959682 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.959589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59714)
0807 15:04:31.959633 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.959662 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:04:31.959792 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
I20260807 15:04:31.962771 15612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 1
I20260807 15:04:31.963340 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.963174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59730)
0807 15:04:31.963251 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.963300 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 15:04:31.963750 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:31.963643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59734)
0807 15:04:31.963693 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:31.963725 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:04:31.963869 15612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
W20260807 15:04:32.012476 15619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:32.012729 15619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:32.012756 15619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:32.014079 15619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:32.014142 15619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:32.014158 15619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:32.014170 15619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:32.014182 15619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:32.016026 15619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:45227,127.1.127.253:39837,127.1.127.252:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:34411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.127.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:32.016359 15619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:32.016644 15619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:32.022501 15625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:32.022545 15624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:32.022503 15627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:32.022869 15619 server_base.cc:1034] running on GCE node
I20260807 15:04:32.023192 15619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:32.023499 15619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:32.024696 15619 hybrid_clock.cc:648] HybridClock initialized: now 1786115072024681 us; error 35 us; skew 500 ppm
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:32.027827 15619 init.cc:377] Logged in from keytab as kudu/127.1.127.252@KRBTEST.COM (short username kudu)
I20260807 15:04:32.028478 15619 webserver.cc:460] Webserver started at http://127.1.127.252:34793/ using document root <none> and password file <none>
I20260807 15:04:32.028751 15619 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:32.028831 15619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:32.029002 15619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:32.030227 15619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data/instance:
uuid: "36ef439e0fe7473688922b8a49944ec9"
format_stamp: "Formatted at 2026-08-07 15:04:32 on dist-test-slave-8tdl"
server_key: "168cd0e09e3c0d3e7de31c95e0c90653"
server_key_iv: "100619970da15776f821d87a02f330c5"
server_key_version: "encryptionkey@0"
I20260807 15:04:32.030659 15619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal/instance:
uuid: "36ef439e0fe7473688922b8a49944ec9"
format_stamp: "Formatted at 2026-08-07 15:04:32 on dist-test-slave-8tdl"
server_key: "168cd0e09e3c0d3e7de31c95e0c90653"
server_key_iv: "100619970da15776f821d87a02f330c5"
server_key_version: "encryptionkey@0"
I20260807 15:04:32.032027 15612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 2
I20260807 15:04:32.032467 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.032318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59742)
0807 15:04:32.032388 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.032429 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:04:32.032513 15619 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 15:04:32.033022 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.032847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59756)
0807 15:04:32.032940 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.032982 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 15:04:32.033157 15612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
I20260807 15:04:32.033541 15634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:32.033764 15619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:32.033854 15619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "36ef439e0fe7473688922b8a49944ec9"
format_stamp: "Formatted at 2026-08-07 15:04:32 on dist-test-slave-8tdl"
server_key: "168cd0e09e3c0d3e7de31c95e0c90653"
server_key_iv: "100619970da15776f821d87a02f330c5"
server_key_version: "encryptionkey@0"
I20260807 15:04:32.033959 15619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:32.034955 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 2
I20260807 15:04:32.035365 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.035245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59772)
0807 15:04:32.035304 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.035340 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:04:32.035758 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.035620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59784)
0807 15:04:32.035685 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.035724 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:04:32.035887 15535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:34411: Network error: Client connection negotiation failed: client connection to 127.1.127.252:34411: connect: Connection refused (error 111)
I20260807 15:04:32.058053 15619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:32.060766 15619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:32.060933 15619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:32.065739 15619 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.252:34411
I20260807 15:04:32.065778 15688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.252:34411 every 8 connection(s)
I20260807 15:04:32.066180 15619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
I20260807 15:04:32.067360 15689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:32.069116 15689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:32.073621 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15619
I20260807 15:04:32.073750 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786114902562993-1535-0/minicluster-data/master-2/wal/instance
I20260807 15:04:32.074025 1535 external_mini_cluster.cc:1428] Setting key 3ca6facab416271457c936bfcae32c79
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:04:32.080639 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.069978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.1.127.254:45227)
0807 15:04:32.070046 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.070048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.070142 (+ 94us) server_negotiation.cc:407] Connection header received
0807 15:04:32.070609 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.070610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.070621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.070751 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.071478 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.072049 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.073106 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.073299 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.073420 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.076413 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.076415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.076415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.076424 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.078939 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.079398 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.079399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.079400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.079499 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.079762 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.079763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.079764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.079932 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.080007 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.080408 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.080501 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:04:32.080955 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.069830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:45227 (local address 127.0.0.1:44444)
0807 15:04:32.070008 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.070053 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:32.070499 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:32.070818 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:32.070822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:32.071171 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:32.071373 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.071378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.072169 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.072170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.072998 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.073000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.073393 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.073394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.073504 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.074228 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:32.074237 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:32.076306 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:32.079039 (+ 2733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.079049 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.079051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.079307 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.079562 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.079563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.079563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.079684 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.080030 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:32.080037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:32.080232 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 15:04:32.080361 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 15:04:32.080524 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260807 15:04:32.081775 15689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:32.084487 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.074517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.1.127.252:34411)
0807 15:04:32.074744 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.074747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.074767 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:04:32.074786 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.074788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.074805 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.074887 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.075593 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.076414 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.077538 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.077870 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.077981 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.080623 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.080631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.080632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.080639 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.082994 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.083425 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.083429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.083429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.083501 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.083775 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.083779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.083779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.083943 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.084219 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.084266 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.084353 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":124,"threads_started":1}
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:32.091197 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.082247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.1.127.253:39837)
0807 15:04:32.082291 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.082293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.082305 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:32.082331 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.082332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.082349 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.082440 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.082755 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.083392 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.084116 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.084337 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.084472 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.086983 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.086984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.086985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.086994 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.089796 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.090266 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.090269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.090269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.090342 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.090567 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.090568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.090568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.090759 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.090829 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.090996 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.091087 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:04:32.091197 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.082116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:39837 (local address 127.0.0.1:46654)
0807 15:04:32.082169 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.082178 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:32.082216 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:32.082448 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:32.082450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:32.082598 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:32.082635 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.082637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.083520 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.083521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.084024 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.084025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.084609 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.084610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.084704 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.085192 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:32.085198 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:32.086882 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:32.089920 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.089921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.089922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.090126 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.090419 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.090420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.090421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.090490 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.090820 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:32.090821 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:32.090874 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:04:32.090967 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:04:32.091087 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:04:32.091897 15689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:04:32.094523 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.085459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.1.127.254:45227)
0807 15:04:32.085506 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.085508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.085536 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:04:32.085567 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.085568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.085576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.085618 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.086074 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.086835 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.087993 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.088306 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.088384 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.091071 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.091073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.091073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.091082 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.092975 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.093403 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.093404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.093404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.093486 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.093795 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.093796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.093797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.094053 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.094290 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.094333 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.094422 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115072, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:04:32.100055 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.092284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59806)
0807 15:04:32.092338 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.092349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:32.092393 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:32.092567 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:32.092568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:32.092727 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:32.092783 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.092785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.093791 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.093793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.094517 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.094521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.095130 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.095131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.095184 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.095751 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:32.095756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:32.097120 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:32.098952 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.098953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.098954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.099131 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.099340 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.099340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.099341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.099399 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.099660 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:32.099661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:32.099713 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:04:32.099796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:04:32.099899 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:04:32.100129 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.092326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.1.127.252:34411)
0807 15:04:32.092366 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.092367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.092380 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:04:32.092473 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.092474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.092485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.092534 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.092866 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.093656 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.094611 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.095050 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.095126 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.097218 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.097220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.097220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.097227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.098860 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.099221 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.099222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.099223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.099275 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.099473 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.099474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.099474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.099593 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.099653 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.099825 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.099948 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":19}
I20260807 15:04:32.101632 15689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: Bootstrap starting.
I20260807 15:04:32.102579 15689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:32.102928 15689 log.cc:824] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:32.103508 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.095356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.1.127.253:39837)
0807 15:04:32.095409 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.095411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.095458 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:04:32.095532 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.095533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.095542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.095586 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.096269 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.096737 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.097512 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.097730 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.097819 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.100495 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.100497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.100497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.100505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.102180 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.102538 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.102539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.102540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.102597 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.102891 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.102892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.102892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.103038 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.103256 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.103300 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.103387 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:04:32.103946 15689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: No bootstrap required, opened a new log
I20260807 15:04:32.105077 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:04:32.105847 15689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.106004 15689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:32.106055 15689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ef439e0fe7473688922b8a49944ec9, State: Initialized, Role: FOLLOWER
I20260807 15:04:32.106207 15689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.106573 15689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:32.106813 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.107131 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:04:32.110666 15713 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:04:32.110823 15713 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:04:32.114400 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.106454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.1.127.253:39837)
0807 15:04:32.106553 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.106555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.106708 (+ 153us) server_negotiation.cc:407] Connection header received
0807 15:04:32.106820 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.106821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.106834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.106867 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.107653 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.108420 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.109196 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.109477 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.109543 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.111229 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.111230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.111231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.111238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.112841 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.113276 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.113278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.113278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.113339 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.113550 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.113551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.113551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.113683 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.113796 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.114220 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.114304 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:04:32.110296 15619 master_runner.cc:186] Error getting master registration for 127.1.127.254:45227: OK, instance_id { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" instance_seqno: 1786115071683174 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:04:32.115285 15619 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:04:32.115680 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.106508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.1.127.254:45227)
0807 15:04:32.106589 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.106591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.107098 (+ 507us) server_negotiation.cc:407] Connection header received
0807 15:04:32.107397 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.107399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.107414 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.107452 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.108292 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.108967 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.109906 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.110215 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.110280 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.111781 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.111782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.111783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.111790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.113681 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.114488 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.114489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.114489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.114551 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.114838 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.114839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.114839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.114967 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.115078 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.115450 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.115585 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:04:32.119508 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.106720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.1.127.252:34411)
0807 15:04:32.106761 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.106763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.107739 (+ 976us) server_negotiation.cc:407] Connection header received
0807 15:04:32.107829 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.107830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.107840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.107880 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.108535 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.109192 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.113772 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.114077 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.114125 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.116501 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.116503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.116503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.116511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.118124 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.118477 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.118478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.118479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.118551 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.118918 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.118919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.118919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.119111 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.119224 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.119329 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.119416 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:04:32.124336 15612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 3
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:04:32.131964 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.124895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.1.127.252:34411)
0807 15:04:32.124988 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.124990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.124999 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:04:32.125021 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.125022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.125031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.125117 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.125431 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.125900 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.126661 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.126881 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.126958 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.129182 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.129183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.129184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.129192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.130654 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.131139 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.131140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.131140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.131202 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.131415 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.131416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.131416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.131537 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.131616 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.131781 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.131855 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:04:32.131964 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.124707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59820)
0807 15:04:32.124823 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.124835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:32.124875 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:32.125123 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:32.125125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:32.125294 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:32.125340 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.125342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.126002 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.126003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.126575 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.126577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.126971 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.126972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.127037 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.127788 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:32.127793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:32.129085 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:32.130846 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.130848 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.130849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.131053 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.131270 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.131271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.131271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.131336 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.131607 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:32.131608 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:32.131657 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:04:32.131742 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:04:32.131855 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 15:04:32.133762 15612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: Bootstrap starting.
I20260807 15:04:32.134752 15612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:32.135062 15535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } attempt: 3
I20260807 15:04:32.135172 15612 log.cc:824] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:32.136420 15612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: No bootstrap required, opened a new log
I20260807 15:04:32.138769 15612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.138942 15612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:32.139000 15612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814a776c366c4e49a7e14142fb929ca7, State: Initialized, Role: FOLLOWER
I20260807 15:04:32.139149 15612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.139470 15716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.139536 15612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:32.139643 15716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 15:04:32 dist-test-slave-8tdl krb5kdc[15452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115071, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
W20260807 15:04:32.141991 15728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:04:32.142120 15728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:04:32.144342 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.135332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:34411 (local address 127.0.0.1:59822)
0807 15:04:32.135394 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:32.135407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:32.135446 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:32.135742 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:32.135743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:32.135901 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:32.135944 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.135946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.136832 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.136833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.137574 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:32.137577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.138238 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.138239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:32.138306 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.138919 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:32.138926 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:32.140925 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:32.143164 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.143165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.143165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.143429 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.143686 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:32.143686 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:32.143687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:32.143744 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:32.144037 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:32.144039 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:32.144088 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:04:32.144137 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:04:32.144243 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:04:32.144764 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:32.135568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.1.127.252:34411)
0807 15:04:32.135606 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:32.135608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:32.135615 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:04:32.135634 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:32.135635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:32.135644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:32.135720 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:32.136032 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.136735 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.137679 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:32.138105 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:32.141075 (+ 2970us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:32.141151 (+ 76us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:32.141152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:32.141153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:32.141158 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:32.143045 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.143531 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.143533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.143533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.143602 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:32.143815 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:32.143815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:32.143816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:32.143957 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:32.144548 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:32.144593 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:04:32.144674 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:04:32.142112 15542 master_runner.cc:186] Error getting master registration for 127.1.127.254:45227: OK, instance_id { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" instance_seqno: 1786115071683174 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:04:32.145560 15542 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:04:32.146202 15535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: Bootstrap starting.
I20260807 15:04:32.146903 15535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:32.147163 15535 log.cc:824] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:32.147909 15535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: No bootstrap required, opened a new log
I20260807 15:04:32.149399 15535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.149540 15535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:32.149571 15535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27aaff21421f4529b3e0ffde21844c59, State: Initialized, Role: FOLLOWER
I20260807 15:04:32.149696 15535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.149960 15731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.150030 15535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:32.150043 15731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:04:32.152408 15742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:04:32.152485 15742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:04:32.154939 15466 master_runner.cc:263] No leader master found from master 27aaff21421f4529b3e0ffde21844c59
I20260807 15:04:32.155004 15466 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 15:04:32.180687 15487 tablet.cc:2369] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:32.200212 15699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:32.200371 15699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.200702 15699 leader_election.cc:288] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27aaff21421f4529b3e0ffde21844c59 (127.1.127.254:45227), 814a776c366c4e49a7e14142fb929ca7 (127.1.127.253:39837)
I20260807 15:04:32.201359 15509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36ef439e0fe7473688922b8a49944ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aaff21421f4529b3e0ffde21844c59" is_pre_election: true
I20260807 15:04:32.201538 15509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ef439e0fe7473688922b8a49944ec9 in term 0.
I20260807 15:04:32.202009 15585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36ef439e0fe7473688922b8a49944ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814a776c366c4e49a7e14142fb929ca7" is_pre_election: true
I20260807 15:04:32.202167 15585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ef439e0fe7473688922b8a49944ec9 in term 0.
I20260807 15:04:32.202419 15636 leader_election.cc:302] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27aaff21421f4529b3e0ffde21844c59, 36ef439e0fe7473688922b8a49944ec9; no voters:
I20260807 15:04:32.202628 15699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:04:32.202718 15699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:04:32.202745 15699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:32.203703 15699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.203856 15699 leader_election.cc:288] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [CANDIDATE]: Term 1 election: Requested vote from peers 27aaff21421f4529b3e0ffde21844c59 (127.1.127.254:45227), 814a776c366c4e49a7e14142fb929ca7 (127.1.127.253:39837)
I20260807 15:04:32.204344 15585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36ef439e0fe7473688922b8a49944ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814a776c366c4e49a7e14142fb929ca7"
I20260807 15:04:32.204473 15585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:32.205925 15585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ef439e0fe7473688922b8a49944ec9 in term 1.
I20260807 15:04:32.206454 15639 leader_election.cc:302] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ef439e0fe7473688922b8a49944ec9, 814a776c366c4e49a7e14142fb929ca7; no voters:
I20260807 15:04:32.206837 15509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36ef439e0fe7473688922b8a49944ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aaff21421f4529b3e0ffde21844c59"
I20260807 15:04:32.206940 15509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:32.208451 15509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ef439e0fe7473688922b8a49944ec9 in term 1.
I20260807 15:04:32.210032 15699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:32.210220 15699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [term 1 LEADER]: Becoming Leader. State: Replica: 36ef439e0fe7473688922b8a49944ec9, State: Running, Role: LEADER
I20260807 15:04:32.210322 15699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } }
I20260807 15:04:32.210891 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36ef439e0fe7473688922b8a49944ec9. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.210994 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:32.211540 15747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:32.211804 15747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:32.214428 15585 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [term 1 FOLLOWER]: Refusing update from remote peer 36ef439e0fe7473688922b8a49944ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:32.214756 15509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [term 1 FOLLOWER]: Refusing update from remote peer 36ef439e0fe7473688922b8a49944ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:32.215216 15699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:32.215677 15699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:32.217274 15731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36ef439e0fe7473688922b8a49944ec9. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.217370 15731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:04:32.219080 15716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36ef439e0fe7473688922b8a49944ec9. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.219192 15716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:04:32.221581 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.221684 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:32.222257 15716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.222349 15716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:04:32.222648 15748 mvcc.cc:205] Tried to move back new op lower bound from 7315927335786930176 to 7315927335774220288. Current Snapshot: MvccSnapshot[applied={T|T < 7315927335786930176}]
I20260807 15:04:32.222934 15751 mvcc.cc:205] Tried to move back new op lower bound from 7315927335786930176 to 7315927335774220288. Current Snapshot: MvccSnapshot[applied={T|T < 7315927335786930176}]
I20260807 15:04:32.223419 15699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.223502 15699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:32.223982 15731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36ef439e0fe7473688922b8a49944ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27aaff21421f4529b3e0ffde21844c59" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 45227 } } peers { permanent_uuid: "814a776c366c4e49a7e14142fb929ca7" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 39837 } } peers { permanent_uuid: "36ef439e0fe7473688922b8a49944ec9" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 34411 } } }
I20260807 15:04:32.224100 15731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:04:32.224854 15749 mvcc.cc:205] Tried to move back new op lower bound from 7315927335786930176 to 7315927335774220288. Current Snapshot: MvccSnapshot[applied={T|T < 7315927335786930176}]
I20260807 15:04:32.227092 15747 catalog_manager.cc:1347] Generated new cluster ID: 5f0dc1eec9dd4b1aa1fd0618bafadc15
I20260807 15:04:32.227149 15747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:04:32.240602 15747 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:04:32.241669 15747 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:32.257252 15747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36ef439e0fe7473688922b8a49944ec9: Generated new TSK 0
I20260807 15:04:32.257625 15747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:04:33.142803 15728 catalog_manager.cc:1259] Loaded cluster ID: 5f0dc1eec9dd4b1aa1fd0618bafadc15
I20260807 15:04:33.142889 15728 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: loading cluster ID for follower catalog manager: success
I20260807 15:04:33.143783 15728 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: acquiring CA information for follower catalog manager: success
I20260807 15:04:33.144143 15728 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 814a776c366c4e49a7e14142fb929ca7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:04:33.152956 15742 catalog_manager.cc:1259] Loaded cluster ID: 5f0dc1eec9dd4b1aa1fd0618bafadc15
I20260807 15:04:33.153074 15742 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: loading cluster ID for follower catalog manager: success
I20260807 15:04:33.153896 15742 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: acquiring CA information for follower catalog manager: success
I20260807 15:04:33.154245 15742 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 27aaff21421f4529b3e0ffde21844c59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:04:33.330248 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15466
I20260807 15:04:33.337936 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15542
I20260807 15:04:33.344894 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15619
2026-08-07T15:04:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3053 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:04:33 dist-test-slave-8tdl krb5kdc[15766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:04:33 dist-test-slave-8tdl krb5kdc[15766](info): set up 2 sockets
Aug 07 15:04:33 dist-test-slave-8tdl krb5kdc[15766](info): commencing operation
krb5kdc: starting...
W20260807 15:04:34.443500 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:04:34.583380 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36481
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:34.684777 15781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:34.685070 15781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:34.685119 15781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:34.686403 15781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:34.686477 15781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:34.686501 15781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:34.686522 15781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:34.686551 15781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:34.688235 15781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:36481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:34.688563 15781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:34.688947 15781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:34.694375 15789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:34.694604 15781 server_base.cc:1034] running on GCE node
W20260807 15:04:34.694375 15787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:34.694414 15786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:34.695044 15781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:34.695358 15781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:34.696528 15781 hybrid_clock.cc:648] HybridClock initialized: now 1786115074696528 us; error 39 us; skew 500 ppm
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115074, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:34.699330 15781 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:34.699875 15781 webserver.cc:460] Webserver started at http://127.1.127.254:46271/ using document root <none> and password file <none>
I20260807 15:04:34.700141 15781 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:34.700219 15781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:34.700416 15781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:34.701610 15781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "9d64650dde934a10a35893be15a7ec9e"
format_stamp: "Formatted at 2026-08-07 15:04:34 on dist-test-slave-8tdl"
server_key: "1d08c38aa77c5a812ee6dafe97b24d88"
server_key_iv: "4d0f9c77d23ac3b3a3834dafc86f286a"
server_key_version: "encryptionkey@0"
I20260807 15:04:34.702031 15781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "9d64650dde934a10a35893be15a7ec9e"
format_stamp: "Formatted at 2026-08-07 15:04:34 on dist-test-slave-8tdl"
server_key: "1d08c38aa77c5a812ee6dafe97b24d88"
server_key_iv: "4d0f9c77d23ac3b3a3834dafc86f286a"
server_key_version: "encryptionkey@0"
I20260807 15:04:34.703661 15781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:04:34.704604 15796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:34.704967 15781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:34.705101 15781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "9d64650dde934a10a35893be15a7ec9e"
format_stamp: "Formatted at 2026-08-07 15:04:34 on dist-test-slave-8tdl"
server_key: "1d08c38aa77c5a812ee6dafe97b24d88"
server_key_iv: "4d0f9c77d23ac3b3a3834dafc86f286a"
server_key_version: "encryptionkey@0"
I20260807 15:04:34.705189 15781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:34.724138 15781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:34.726372 15781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:34.726584 15781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:34.731215 15781 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:36481
I20260807 15:04:34.731256 15849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:36481 every 8 connection(s)
I20260807 15:04:34.731633 15781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:34.732632 15850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:34.735167 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e: Bootstrap starting.
I20260807 15:04:34.735860 15850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:34.736167 15850 log.cc:824] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:34.737084 15850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e: No bootstrap required, opened a new log
I20260807 15:04:34.738987 15850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } }
I20260807 15:04:34.739149 15850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:34.739173 15850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d64650dde934a10a35893be15a7ec9e, State: Initialized, Role: FOLLOWER
I20260807 15:04:34.739360 15850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } }
I20260807 15:04:34.739429 15850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:34.739452 15850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:34.739506 15850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:34.739962 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15781
I20260807 15:04:34.740070 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:04:34.740286 1535 external_mini_cluster.cc:1428] Setting key 3722e9a08d5670ab04ccf0d4bd9867a2
I20260807 15:04:34.740365 15850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } }
I20260807 15:04:34.740487 15850 leader_election.cc:302] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d64650dde934a10a35893be15a7ec9e; no voters:
I20260807 15:04:34.740742 15850 leader_election.cc:288] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:04:34.740779 15853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:34.740919 15853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [term 1 LEADER]: Becoming Leader. State: Replica: 9d64650dde934a10a35893be15a7ec9e, State: Running, Role: LEADER
I20260807 15:04:34.740998 15853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } }
I20260807 15:04:34.741047 15850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:34.741345 15853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d64650dde934a10a35893be15a7ec9e. Latest consensus state: current_term: 1 leader_uuid: "9d64650dde934a10a35893be15a7ec9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } } }
I20260807 15:04:34.741433 15853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:34.741717 15860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:34.741688 15855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d64650dde934a10a35893be15a7ec9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d64650dde934a10a35893be15a7ec9e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36481 } } }
I20260807 15:04:34.741858 15855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:34.742599 15860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:34.744443 15860 catalog_manager.cc:1347] Generated new cluster ID: 78ff7ddd36114b258afabf4b2907f8c5
I20260807 15:04:34.744509 15860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:34.752785 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:34.740875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.1.127.254:36481)
0807 15:04:34.742008 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:34.742019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:34.742058 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:04:34.742220 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:34.742227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:34.742293 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:34.742457 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:34.743371 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:34.744647 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:34.745450 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:34.745921 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:34.746047 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:34.748592 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:34.748596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:34.748596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:34.748611 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:34.751074 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:34.751501 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:34.751504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:34.751505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:34.751625 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:34.751849 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:34.751850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:34.751850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:34.752124 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:34.752311 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:34.752395 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:04:34.752557 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1068,"thread_start_us":91,"threads_started":1}
I20260807 15:04:34.753865 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:04:34.757592 15860 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:04:34.758186 15860 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:34.770773 15860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d64650dde934a10a35893be15a7ec9e: Generated new TSK 0
I20260807 15:04:34.771056 15860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:04:34.808506 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15781
2026-08-07T15:04:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1461 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15878](info): set up 2 sockets
Aug 07 15:04:34 dist-test-slave-8tdl krb5kdc[15878](info): commencing operation
krb5kdc: starting...
W20260807 15:04:35.910254 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:04:36.050410 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:36.140851 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:36.141141 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:36.141183 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:36.142409 15892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:36.142494 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:36.142518 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:36.142575 15892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:36.142599 15892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:36.144158 15892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:36.144461 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:36.144708 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:36.150041 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.150068 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.150043 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:36.150452 15892 server_base.cc:1034] running on GCE node
I20260807 15:04:36.150791 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:36.151242 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:36.152452 15892 hybrid_clock.cc:648] HybridClock initialized: now 1786115076152428 us; error 44 us; skew 500 ppm
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:36.160374 15892 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:36.160872 15892 webserver.cc:460] Webserver started at http://127.1.127.254:35355/ using document root <none> and password file <none>
I20260807 15:04:36.161093 15892 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:36.161144 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:36.161247 15892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:36.162488 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "f1d8c43900c64dba9a93c416744e6aa0"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "13fba8c8b4973b9f7c79763b9e1efb68"
server_key_iv: "4394797a2ae85c9f974787808f3b2c07"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.162945 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "f1d8c43900c64dba9a93c416744e6aa0"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "13fba8c8b4973b9f7c79763b9e1efb68"
server_key_iv: "4394797a2ae85c9f974787808f3b2c07"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.164788 15892 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:04:36.165670 15907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.165896 15892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:36.166033 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "f1d8c43900c64dba9a93c416744e6aa0"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "13fba8c8b4973b9f7c79763b9e1efb68"
server_key_iv: "4394797a2ae85c9f974787808f3b2c07"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.166134 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:36.176617 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:36.179181 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:36.179376 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:36.184010 15892 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41887
I20260807 15:04:36.184052 15959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41887 every 8 connection(s)
I20260807 15:04:36.184458 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:36.185509 15960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:36.187582 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15892
I20260807 15:04:36.187726 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:04:36.187978 1535 external_mini_cluster.cc:1428] Setting key 39d182e29ebd11b556535c11b434d142
I20260807 15:04:36.188501 15960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap starting.
I20260807 15:04:36.189267 15960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:36.189677 15960 log.cc:824] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:36.190554 15960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: No bootstrap required, opened a new log
I20260807 15:04:36.192823 15960 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:36.193048 15960 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:36.193090 15960 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Initialized, Role: FOLLOWER
I20260807 15:04:36.193274 15960 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:36.193399 15960 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:36.193442 15960 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:36.193504 15960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:36.194380 15960 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:36.194520 15960 leader_election.cc:302] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d8c43900c64dba9a93c416744e6aa0; no voters:
I20260807 15:04:36.194767 15960 leader_election.cc:288] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:04:36.194989 15966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:36.195118 15960 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:36.195161 15966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 LEADER]: Becoming Leader. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Running, Role: LEADER
I20260807 15:04:36.195286 15966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:36.195627 15966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d8c43900c64dba9a93c416744e6aa0. Latest consensus state: current_term: 1 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:36.195708 15966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:36.195895 15967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:36.195960 15967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:36.196079 15973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:36.197301 15973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:36.199512 15973 catalog_manager.cc:1347] Generated new cluster ID: d15fb114a9aa4ebc96fa7c1f312b2004
I20260807 15:04:36.199571 15973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:04:36.200877 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.188652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.1.127.254:41887)
0807 15:04:36.188854 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.188862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.188893 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:04:36.189355 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.189362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.189464 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.189601 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:36.190430 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.191499 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.192673 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.193242 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.193338 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.196640 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:36.196645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:36.196645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:36.196660 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:36.198749 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.199233 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.199237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.199237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.199360 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.199638 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.199640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.199640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.200011 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:36.200152 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.200493 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.200649 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260807 15:04:36.213762 15973 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:04:36.214615 15973 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:36.227090 15973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Generated new TSK 0
I20260807 15:04:36.227370 15973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:04:36.264398 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:36.356643 15986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:36.356912 15986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:36.356961 15986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:36.358263 15986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:36.358341 15986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:36.358394 15986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:36.360699 15986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:36.360998 15986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:36.361276 15986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:04:36.367020 15986 server_base.cc:1034] running on GCE node
W20260807 15:04:36.366995 15991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.367060 15992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.366989 15994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:36.367445 15986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:36.367797 15986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:36.368984 15986 hybrid_clock.cc:648] HybridClock initialized: now 1786115076368968 us; error 29 us; skew 500 ppm
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:36.371976 15986 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:36.372514 15986 webserver.cc:460] Webserver started at http://127.1.127.193:40845/ using document root <none> and password file <none>
I20260807 15:04:36.372790 15986 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:36.372866 15986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:36.373025 15986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:36.374148 15986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "ef05c06d2810450aa788ab5f574bd9ae"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7db86ffdca5ed63cb0e4446e0b502302"
server_key_iv: "16f57ba515a009a90a2f3681b7129bcc"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.374536 15986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "ef05c06d2810450aa788ab5f574bd9ae"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7db86ffdca5ed63cb0e4446e0b502302"
server_key_iv: "16f57ba515a009a90a2f3681b7129bcc"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.376107 15986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:36.377071 16001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.377308 15986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:36.377426 15986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "ef05c06d2810450aa788ab5f574bd9ae"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7db86ffdca5ed63cb0e4446e0b502302"
server_key_iv: "16f57ba515a009a90a2f3681b7129bcc"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.377507 15986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:36.386507 15986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:36.389081 15986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:36.389267 15986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:36.389530 15986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:36.389887 15986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:36.389947 15986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.389994 15986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:36.390023 15986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.397408 15986 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35291
I20260807 15:04:36.397491 16117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35291 every 8 connection(s)
I20260807 15:04:36.397861 15986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:36.400859 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 15986
I20260807 15:04:36.401002 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:04:36.401206 1535 external_mini_cluster.cc:1428] Setting key 579245d7e074fc169ace6e44217a0928
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:36.410604 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.398833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:45839 (local address 127.1.127.254:41887)
0807 15:04:36.398937 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.398940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.399312 (+ 372us) server_negotiation.cc:407] Connection header received
0807 15:04:36.399848 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.399850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.399862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.399924 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:36.401456 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.402285 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.403443 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.403681 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.403763 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.406380 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:36.406382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:36.406382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:36.406392 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:36.408477 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.409219 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.409221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.409221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.409302 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.409740 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.409742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.409742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.409922 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:36.410002 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.410354 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.410468 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:04:36.410845 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.398854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:45839)
0807 15:04:36.399147 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.399189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.399724 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.400061 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.400064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.400986 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:36.401318 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.401326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.402382 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.402383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.403334 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.403337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.403790 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.403791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.403921 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.404549 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:36.404559 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:36.406255 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:36.408617 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.408629 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.408631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.409119 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.409428 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.409430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.409430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.409587 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.410037 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:36.410045 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:36.410195 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.410302 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.410503 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":125,"threads_started":1}
I20260807 15:04:36.411546 16118 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:36.411677 16118 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:36.411888 16118 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:36.412627 15924 ts_manager.cc:194] Registered new tserver with Master: ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193:35291)
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:04:36.413388 15924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:45839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:04:36.435669 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:36.523840 16123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:36.524140 16123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:36.524184 16123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:36.525429 16123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:36.525502 16123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:36.525548 16123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:36.527122 16123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:36.527436 16123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:36.527691 16123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:04:36.533392 16123 server_base.cc:1034] running on GCE node
W20260807 15:04:36.533420 16129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.533565 16131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.533361 16128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:36.533849 16123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:36.534215 16123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:36.535390 16123 hybrid_clock.cc:648] HybridClock initialized: now 1786115076535366 us; error 42 us; skew 500 ppm
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:36.538352 16123 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:36.539036 16123 webserver.cc:460] Webserver started at http://127.1.127.194:46245/ using document root <none> and password file <none>
I20260807 15:04:36.539351 16123 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:36.539443 16123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:36.539690 16123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:36.540992 16123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "7d6de7dc0fa242cd83d951f89146373b"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7cf87ddbd9d4a482928df48d28acdc36"
server_key_iv: "e2cc4113bda825ab3646f3b78bf8f8be"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.541424 16123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "7d6de7dc0fa242cd83d951f89146373b"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7cf87ddbd9d4a482928df48d28acdc36"
server_key_iv: "e2cc4113bda825ab3646f3b78bf8f8be"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.543092 16123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:04:36.544027 16138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.544327 16123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:36.544553 16123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "7d6de7dc0fa242cd83d951f89146373b"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7cf87ddbd9d4a482928df48d28acdc36"
server_key_iv: "e2cc4113bda825ab3646f3b78bf8f8be"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.544663 16123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:36.558578 16123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:36.560926 16123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:36.561142 16123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:36.561460 16123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:36.561805 16123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:36.561838 16123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.561894 16123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:36.561923 16123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.568748 16123 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45523
I20260807 15:04:36.568902 16253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45523 every 8 connection(s)
I20260807 15:04:36.569197 16123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:36.572510 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16123
I20260807 15:04:36.572629 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:04:36.572834 1535 external_mini_cluster.cc:1428] Setting key 56d257f1f3fe8ea8b8a7dea70286f61c
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:36.581315 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.569989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57787 (local address 127.1.127.254:41887)
0807 15:04:36.570057 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.570059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.570339 (+ 280us) server_negotiation.cc:407] Connection header received
0807 15:04:36.571008 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.571009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.571021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.571076 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:36.572081 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.572893 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.574219 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.574557 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.574641 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.577435 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:36.577437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:36.577437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:36.577446 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:36.579350 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.579816 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.579854 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.579855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.579931 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.580283 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.580284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.580285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.580459 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:36.580554 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.581115 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.581216 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:04:36.581641 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.569978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:57787)
0807 15:04:36.570177 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.570216 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.570895 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.571141 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.571145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.571745 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:36.571964 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.571970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.573055 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.573056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.574107 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.574111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.574734 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.574736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.574888 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.575497 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:36.575507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:36.577324 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:36.579467 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.579475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.579478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.579729 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.580044 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.580045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.580046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.580174 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.580564 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:36.580573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:36.580767 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.581065 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.581261 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260807 15:04:36.582326 16254 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:36.582437 16254 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:36.582712 16254 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:36.583331 15924 ts_manager.cc:194] Registered new tserver with Master: 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:04:36.583860 15924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:57787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:04:36.606590 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:36.693141 16259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:36.693385 16259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:36.693413 16259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:36.694907 16259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:36.694965 16259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:36.694998 16259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:36.696774 16259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:36.697077 16259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:36.697300 16259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:36.702992 16265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.702909 16267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:36.702950 16264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:36.702925 16259 server_base.cc:1034] running on GCE node
I20260807 15:04:36.703475 16259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:36.703747 16259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:36.704931 16259 hybrid_clock.cc:648] HybridClock initialized: now 1786115076704906 us; error 63 us; skew 500 ppm
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:36.707609 16259 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:36.708124 16259 webserver.cc:460] Webserver started at http://127.1.127.195:36875/ using document root <none> and password file <none>
I20260807 15:04:36.708372 16259 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:36.708443 16259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:36.708604 16259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:36.709712 16259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "87ed77a16918afce5b5798e042d929bf"
server_key_iv: "e43a1c450dc9fe2aef8884d13a4eae2f"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.710107 16259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "87ed77a16918afce5b5798e042d929bf"
server_key_iv: "e43a1c450dc9fe2aef8884d13a4eae2f"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.711663 16259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:04:36.712492 16274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.712751 16259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:36.712860 16259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "87ed77a16918afce5b5798e042d929bf"
server_key_iv: "e43a1c450dc9fe2aef8884d13a4eae2f"
server_key_version: "encryptionkey@0"
I20260807 15:04:36.712939 16259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:36.728122 16259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:36.730796 16259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:36.730979 16259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:36.731240 16259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:36.731570 16259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:36.731626 16259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.731671 16259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:36.731702 16259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:36.738400 16259 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38495
I20260807 15:04:36.738453 16390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38495 every 8 connection(s)
I20260807 15:04:36.738914 16259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:36.742846 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16259
I20260807 15:04:36.743031 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:04:36.743196 1535 external_mini_cluster.cc:1428] Setting key adc75d8b433285e4717db2ca68f30395
Aug 07 15:04:36 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:36.750207 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.740001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:50529 (local address 127.1.127.254:41887)
0807 15:04:36.740070 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.740072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.740085 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:04:36.740687 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.740688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.740700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.740766 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:36.741925 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.742474 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.744072 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.744331 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.744393 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.746859 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:36.746861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:36.746861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:36.746869 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:36.748526 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.749109 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.749111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.749111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.749172 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.749466 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.749466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.749466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.749595 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:36.749655 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.750045 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.750125 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:04:36.750397 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.739621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:50529)
0807 15:04:36.739860 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.739926 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.740553 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.740885 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.740890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.741522 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:36.741757 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.741762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.742576 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.742577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.743978 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.743981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.744465 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.744466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.744583 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.745086 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:36.745095 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:36.746753 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:36.748628 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.748638 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.748640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.749010 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.749271 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:36.749272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:36.749272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:36.749395 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:36.749674 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:36.749680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:36.749826 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.749991 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.750147 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260807 15:04:36.751116 16392 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:36.751261 16392 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:36.751494 16392 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:36.752072 15924 ts_manager.cc:194] Registered new tserver with Master: 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495)
I20260807 15:04:36.752535 15924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:50529
I20260807 15:04:36.755312 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:04:36.763495 15963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.756823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.1.127.254:41887)
0807 15:04:36.756883 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.756885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.757010 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:04:36.757079 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.757080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.757090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.757132 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:36.757674 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.758182 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.759020 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.759224 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.759280 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.760596 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:36.760597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:36.760597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:36.760603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:36.762143 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.762541 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.762543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.762544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.762610 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:36.762856 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:36.762857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:36.762857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:36.763028 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:36.763142 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.763299 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.763386 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:04:36.765291 15924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:04:36.767175 15924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:04:36.774662 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.769596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35291 (local address 127.0.0.1:34300)
0807 15:04:36.769928 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.769941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.770011 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.770383 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.770387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.770390 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:36.770465 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.770468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.771947 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.771948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.773162 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.773165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.774319 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.774411 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.774426 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.774508 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.774578 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":9,"thread_start_us":98,"threads_started":1}
I20260807 15:04:36.774639 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.769705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.1.127.193:35291)
0807 15:04:36.770040 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.770044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.770056 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:36.770098 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.770100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.770265 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.770360 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:36.770591 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.771793 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.773264 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.774222 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774313 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.774380 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.774521 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.774558 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":91,"threads_started":1}
I20260807 15:04:36.775485 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.769592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38495 (local address 127.0.0.1:33360)
0807 15:04:36.769770 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.769817 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.769886 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.770845 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.770846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.770848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:36.770916 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.770918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.771796 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.771798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.773162 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.773165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.775177 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.775178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.775269 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.775275 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.775342 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.775392 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":11,"thread_start_us":65,"threads_started":1}
I20260807 15:04:36.775504 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.769679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.1.127.195:38495)
0807 15:04:36.770550 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.770555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.770632 (+ 77us) server_negotiation.cc:407] Connection header received
0807 15:04:36.770704 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.770707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.770722 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.770859 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:36.770974 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.771679 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.773264 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.774222 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774313 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.774381 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.775367 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.775420 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":106,"threads_started":1}
I20260807 15:04:36.775677 16052 tablet_service.cc:1467] Processing CreateTablet for tablet f9d9f96061b54e98a61d610029ac32c0 (DEFAULT_TABLE table=test-table [id=248775e5660f458c97e43d51d6bca9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:36.776099 16052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d9f96061b54e98a61d610029ac32c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:36.776258 16323 tablet_service.cc:1467] Processing CreateTablet for tablet f9d9f96061b54e98a61d610029ac32c0 (DEFAULT_TABLE table=test-table [id=248775e5660f458c97e43d51d6bca9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:36.776595 16323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d9f96061b54e98a61d610029ac32c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:36.778929 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.770038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.1.127.194:45523)
0807 15:04:36.770211 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.770215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.770227 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:36.770270 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.770272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.770288 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.770415 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:36.770619 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.771793 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774078 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.774924 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774997 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.775051 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.778809 (+ 3758us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.778860 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260807 15:04:36.779162 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.769813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45523 (local address 127.0.0.1:36818)
0807 15:04:36.769953 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.769962 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.770054 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.770497 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.770497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.770500 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:36.770537 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.770539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.773269 (+ 2730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.773270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.773983 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.773985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.774998 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.774999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.775071 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.775075 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.775129 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.779088 (+ 3959us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260807 15:04:36.779816 16410 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap starting.
I20260807 15:04:36.779969 16188 tablet_service.cc:1467] Processing CreateTablet for tablet f9d9f96061b54e98a61d610029ac32c0 (DEFAULT_TABLE table=test-table [id=248775e5660f458c97e43d51d6bca9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:36.780156 16411 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Bootstrap starting.
I20260807 15:04:36.780369 16188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d9f96061b54e98a61d610029ac32c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:36.780581 16410 tablet_bootstrap.cc:654] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:36.780737 16411 tablet_bootstrap.cc:654] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:36.781162 16410 log.cc:824] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:36.781234 16411 log.cc:824] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:36.781972 16410 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: No bootstrap required, opened a new log
I20260807 15:04:36.782023 16410 ts_tablet_manager.cc:1397] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:04:36.782038 16411 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: No bootstrap required, opened a new log
I20260807 15:04:36.782102 16411 ts_tablet_manager.cc:1397] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:04:36.783486 16410 raft_consensus.cc:348] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.783622 16410 raft_consensus.cc:374] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:36.783643 16410 raft_consensus.cc:729] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Initialized, Role: FOLLOWER
I20260807 15:04:36.783720 16410 consensus_queue.cc:260] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.784020 16410 ts_tablet_manager.cc:1428] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:04:36.784042 16415 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Bootstrap starting.
I20260807 15:04:36.784173 16118 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
I20260807 15:04:36.784260 16411 raft_consensus.cc:348] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.784462 16411 raft_consensus.cc:374] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:36.784502 16411 raft_consensus.cc:729] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4ee237c34f47cd8e9d949b4749a8e5, State: Initialized, Role: FOLLOWER
I20260807 15:04:36.784647 16411 consensus_queue.cc:260] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.784876 16415 tablet_bootstrap.cc:654] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:36.784965 16411 ts_tablet_manager.cc:1428] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:04:36.785094 16392 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
I20260807 15:04:36.785576 16415 log.cc:824] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:36.786597 16415 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: No bootstrap required, opened a new log
I20260807 15:04:36.786664 16415 ts_tablet_manager.cc:1397] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:04:36.788838 16415 raft_consensus.cc:348] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.789003 16415 raft_consensus.cc:374] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:36.789027 16415 raft_consensus.cc:729] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de7dc0fa242cd83d951f89146373b, State: Initialized, Role: FOLLOWER
I20260807 15:04:36.789166 16415 consensus_queue.cc:260] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.789494 16415 ts_tablet_manager.cc:1428] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:04:36.789659 16254 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
W20260807 15:04:36.819908 16255 tablet.cc:2369] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:36.883822 16414 raft_consensus.cc:482] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:36.884066 16414 raft_consensus.cc:504] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.884529 16414 leader_election.cc:288] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
I20260807 15:04:36.888268 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.884950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:57179 (local address 127.1.127.194:45523)
0807 15:04:36.884996 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.884998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.885087 (+ 89us) server_negotiation.cc:407] Connection header received
0807 15:04:36.885142 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.885143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.885154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.885216 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:36.885373 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.886098 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887462 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.887875 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887964 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.887974 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.888145 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.888190 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:04:36.888270 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.884828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:60753 (local address 127.1.127.195:38495)
0807 15:04:36.884921 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.884923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.884936 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:04:36.884972 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.884973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.884985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.885100 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:36.885298 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.886081 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887491 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.887873 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887972 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.887988 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.888155 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.888197 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:04:36.888281 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.884716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38495 (local address 127.1.127.193:60753)
0807 15:04:36.884781 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.884795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.884845 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.885108 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.885109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.885111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:36.885211 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.885213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.886262 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.886264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.887350 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.887354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.887972 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.888075 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.888078 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.888149 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.888198 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":55}
I20260807 15:04:36.888285 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.884857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45523 (local address 127.1.127.193:57179)
0807 15:04:36.884993 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.885002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:36.885082 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:36.885231 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:36.885231 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:36.885233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:36.885304 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.885305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.886262 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.886264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.887357 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:36.887362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.887976 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.887978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:36.888056 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.888063 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:04:36.888137 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:04:36.888187 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":68,"thread_start_us":88,"threads_started":1}
I20260807 15:04:36.888733 16343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d9f96061b54e98a61d610029ac32c0" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" is_pre_election: true
I20260807 15:04:36.888731 16208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d9f96061b54e98a61d610029ac32c0" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de7dc0fa242cd83d951f89146373b" is_pre_election: true
I20260807 15:04:36.888913 16343 raft_consensus.cc:2454] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 0.
I20260807 15:04:36.888926 16208 raft_consensus.cc:2454] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 0.
I20260807 15:04:36.889176 16002 leader_election.cc:302] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de7dc0fa242cd83d951f89146373b, ef05c06d2810450aa788ab5f574bd9ae; no voters:
I20260807 15:04:36.889333 16414 raft_consensus.cc:2781] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:04:36.889428 16414 raft_consensus.cc:482] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:04:36.889468 16414 raft_consensus.cc:3037] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:36.890252 16414 raft_consensus.cc:504] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.890422 16414 leader_election.cc:288] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 election: Requested vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
I20260807 15:04:36.890621 16343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d9f96061b54e98a61d610029ac32c0" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
I20260807 15:04:36.890648 16208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d9f96061b54e98a61d610029ac32c0" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de7dc0fa242cd83d951f89146373b"
I20260807 15:04:36.890748 16343 raft_consensus.cc:3037] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:36.890761 16208 raft_consensus.cc:3037] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:36.891575 16208 raft_consensus.cc:2454] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 1.
I20260807 15:04:36.891589 16343 raft_consensus.cc:2454] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 1.
I20260807 15:04:36.891750 16002 leader_election.cc:302] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de7dc0fa242cd83d951f89146373b, ef05c06d2810450aa788ab5f574bd9ae; no voters:
I20260807 15:04:36.891877 16414 raft_consensus.cc:2781] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:36.892067 16414 raft_consensus.cc:686] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 LEADER]: Becoming Leader. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Running, Role: LEADER
I20260807 15:04:36.892196 16414 consensus_queue.cc:237] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:36.893141 15923 catalog_manager.cc:5495] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae reported cstate change: term changed from 0 to 1, leader changed from <none> to ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193). New cstate: current_term: 1 leader_uuid: "ef05c06d2810450aa788ab5f574bd9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } health_report { overall_health: UNKNOWN } } }
W20260807 15:04:36.898775 16119 tablet.cc:2369] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:36.921077 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.918441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.1.127.193:35291)
0807 15:04:36.918511 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:36.918513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:36.918521 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:04:36.918568 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:36.918569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:36.918589 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:36.918672 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:04:36.918848 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.919333 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.920181 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:36.920383 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:36.920501 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:36.920685 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:04:36.920793 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:04:36.920887 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:36.920929 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:04:36.920954 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:04:36.923133 16343 raft_consensus.cc:1261] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Refusing update from remote peer ef05c06d2810450aa788ab5f574bd9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:36.923184 16208 raft_consensus.cc:1261] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Refusing update from remote peer ef05c06d2810450aa788ab5f574bd9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:36.923584 16420 consensus_queue.cc:1035] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:36.923771 16414 consensus_queue.cc:1035] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:36.927235 16424 mvcc.cc:205] Tried to move back new op lower bound from 7315927355073982464 to 7315927354951340032. Current Snapshot: MvccSnapshot[applied={T|T < 7315927355073982464 or (T in {7315927355073982464})}]
I20260807 15:04:36.929195 15924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:36.929307 15924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:36.930115 15924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Sending DeleteTablet for 3 replicas of tablet f9d9f96061b54e98a61d610029ac32c0
I20260807 15:04:36.930474 16052 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d9f96061b54e98a61d610029ac32c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:36 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:34300
I20260807 15:04:36.930612 16323 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d9f96061b54e98a61d610029ac32c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:36 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:33360
I20260807 15:04:36.930612 16188 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d9f96061b54e98a61d610029ac32c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:36 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:36818
I20260807 15:04:36.930702 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15986
I20260807 15:04:36.932173 16435 tablet_replica.cc:330] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: stopping tablet replica
I20260807 15:04:36.932323 16435 raft_consensus.cc:2229] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:36.932457 16435 raft_consensus.cc:2258] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:36.932653 16434 tablet_replica.cc:330] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: stopping tablet replica
I20260807 15:04:36.932763 16434 raft_consensus.cc:2229] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:36.932869 16434 raft_consensus.cc:2258] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:36.933130 16435 ts_tablet_manager.cc:1905] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:36.933310 16434 ts_tablet_manager.cc:1905] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:36.935176 16435 ts_tablet_manager.cc:1918] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:36.935289 16435 log.cc:1196] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/f9d9f96061b54e98a61d610029ac32c0
I20260807 15:04:36.935405 16434 ts_tablet_manager.cc:1918] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:36.935490 16434 log.cc:1196] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/f9d9f96061b54e98a61d610029ac32c0
I20260807 15:04:36.935660 16435 ts_tablet_manager.cc:1939] T f9d9f96061b54e98a61d610029ac32c0 P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting consensus metadata
I20260807 15:04:36.935788 16434 ts_tablet_manager.cc:1939] T f9d9f96061b54e98a61d610029ac32c0 P 7d6de7dc0fa242cd83d951f89146373b: Deleting consensus metadata
I20260807 15:04:36.936229 15908 catalog_manager.cc:4841] TS 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): tablet f9d9f96061b54e98a61d610029ac32c0 (table test-table [id=248775e5660f458c97e43d51d6bca9cb]) successfully deleted
I20260807 15:04:36.936357 15908 catalog_manager.cc:4841] TS 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): tablet f9d9f96061b54e98a61d610029ac32c0 (table test-table [id=248775e5660f458c97e43d51d6bca9cb]) successfully deleted
W20260807 15:04:36.939486 15909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:35291 (error 108)
I20260807 15:04:36.939738 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16123
I20260807 15:04:36.946745 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16259
I20260807 15:04:36.947079 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:36.946831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35291 (local address 127.0.0.1:34324)
0807 15:04:36.946972 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:36.947026 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260807 15:04:36.948298 15909 catalog_manager.cc:4584] TS ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193:35291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9d9f96061b54e98a61d610029ac32c0: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35291: connect: Connection refused (error 111)
I20260807 15:04:36.954704 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 15892
I20260807 15:04:36.963037 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--webserver_interface=127.1.127.254
--webserver_port=35355
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:37.053579 16437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:37.053870 16437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:37.053913 16437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:37.055223 16437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:37.055305 16437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:37.055332 16437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:37.055353 16437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:37.055382 16437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:37.057035 16437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:37.057337 16437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:37.057592 16437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:37.067840 16447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.067986 16444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.067843 16443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:37.068691 16437 server_base.cc:1034] running on GCE node
I20260807 15:04:37.069033 16437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:37.069434 16437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:37.070627 16437 hybrid_clock.cc:648] HybridClock initialized: now 1786115077070603 us; error 44 us; skew 500 ppm
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:37.073779 16437 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:37.074337 16437 webserver.cc:460] Webserver started at http://127.1.127.254:35355/ using document root <none> and password file <none>
I20260807 15:04:37.074608 16437 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:37.074735 16437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:37.076292 16437 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:37.077255 16455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.077538 16437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:37.077682 16437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "f1d8c43900c64dba9a93c416744e6aa0"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "13fba8c8b4973b9f7c79763b9e1efb68"
server_key_iv: "4394797a2ae85c9f974787808f3b2c07"
server_key_version: "encryptionkey@0"
I20260807 15:04:37.078153 16437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:37.087915 16437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:37.090773 16437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:37.090969 16437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:37.096596 16437 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41887
I20260807 15:04:37.096647 16510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41887 every 8 connection(s)
I20260807 15:04:37.097131 16437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:37.099104 16511 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:04:37.099737 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16437
I20260807 15:04:37.100183 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:35291
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=40845
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:37.100311 16511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap starting.
I20260807 15:04:37.111469 16511 log.cc:824] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:37.113951 16511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:04:37.114331 16511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap complete.
I20260807 15:04:37.117245 16511 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:37.117522 16511 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Initialized, Role: FOLLOWER
I20260807 15:04:37.118005 16511 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:37.118106 16511 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:37.118180 16511 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:37.118237 16511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:04:37.119619 16511 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:37.119728 16511 leader_election.cc:302] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d8c43900c64dba9a93c416744e6aa0; no voters:
I20260807 15:04:37.119985 16511 leader_election.cc:288] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:04:37.120028 16515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:04:37.120149 16515 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 LEADER]: Becoming Leader. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Running, Role: LEADER
I20260807 15:04:37.120255 16515 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:37.120307 16511 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:37.120586 16517 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:37.120678 16517 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:37.120944 16516 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d8c43900c64dba9a93c416744e6aa0. Latest consensus state: current_term: 2 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:37.121076 16516 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:37.121223 16520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:37.122159 16520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:37.122279 16520 catalog_manager.cc:1259] Loaded cluster ID: d15fb114a9aa4ebc96fa7c1f312b2004
I20260807 15:04:37.122316 16520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:04:37.122615 16520 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 15:04:37.125784 16520 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 15:04:37.128118 16520 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:37.128379 16520 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Loaded TSK: 0
I20260807 15:04:37.128612 16520 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:04:37.205960 16513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:37.206200 16513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:37.206229 16513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:37.207664 16513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:37.207724 16513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:37.207755 16513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:37.209438 16513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:35291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=40845
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:37.209725 16513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:37.210017 16513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:37.215746 16540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.215749 16539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.215749 16542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:37.216310 16513 server_base.cc:1034] running on GCE node
I20260807 15:04:37.216583 16513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:37.216902 16513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:37.218112 16513 hybrid_clock.cc:648] HybridClock initialized: now 1786115077218096 us; error 30 us; skew 500 ppm
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:37.221963 16513 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:37.222631 16513 webserver.cc:460] Webserver started at http://127.1.127.193:40845/ using document root <none> and password file <none>
I20260807 15:04:37.222949 16513 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:37.223040 16513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:37.225028 16513 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:04:37.226042 16549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.226277 16513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:37.226374 16513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "ef05c06d2810450aa788ab5f574bd9ae"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7db86ffdca5ed63cb0e4446e0b502302"
server_key_iv: "16f57ba515a009a90a2f3681b7129bcc"
server_key_version: "encryptionkey@0"
I20260807 15:04:37.226994 16513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:37.249064 16513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:37.252058 16513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:37.252193 16513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:37.252521 16513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:37.253048 16556 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:04:37.255328 16513 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:04:37.255421 16513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260807 15:04:37.255488 16513 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:04:37.256268 16513 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:04:37.256332 16513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:37.256399 16556 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap starting.
I20260807 15:04:37.264636 16513 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35291
I20260807 15:04:37.265075 16513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:37.267488 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16513
I20260807 15:04:37.267810 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:45523
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=46245
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:37.269129 16556 log.cc:824] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:37.269506 16667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35291 every 8 connection(s)
I20260807 15:04:37.272090 16556 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:04:37.272455 16556 tablet_bootstrap.cc:492] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap complete.
I20260807 15:04:37.273998 16556 ts_tablet_manager.cc:1397] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.000s
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:37.275559 16556 raft_consensus.cc:348] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:37.275910 16556 raft_consensus.cc:729] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Initialized, Role: FOLLOWER
I20260807 15:04:37.277465 16556 consensus_queue.cc:260] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } }
I20260807 15:04:37.277865 16556 ts_tablet_manager.cc:1428] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260807 15:04:37.280499 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.265868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:45773)
0807 15:04:37.266176 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.266212 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.266801 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.269627 (+ 2826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.269632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.269983 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:37.270192 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.270198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.271536 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.271537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.272510 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.272514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.273160 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.273161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.273310 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.273918 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:37.273926 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:37.275628 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:37.278147 (+ 2519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.278157 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.278159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.278516 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.278895 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.278896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.278896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.279060 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.279644 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:37.279653 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:37.279833 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.279960 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.280198 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":103,"threads_started":1}
I20260807 15:04:37.281232 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.265932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:45773 (local address 127.1.127.254:41887)
0807 15:04:37.266866 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.266876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.266926 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:04:37.267068 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.267073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.267116 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.267276 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:37.270354 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.271367 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.272606 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.273035 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.273168 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.275791 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:37.275796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:37.275796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:37.275809 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:37.278014 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.278622 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.278627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.278627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.278816 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.279136 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.279137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.279137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.279516 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:37.279610 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.280733 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.280915 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":82,"threads_started":1}
I20260807 15:04:37.282058 16668 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:37.282179 16668 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:37.282389 16668 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:37.283330 16473 ts_manager.cc:194] Registered new tserver with Master: ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193:35291)
I20260807 15:04:37.284574 16473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:45773
I20260807 15:04:37.285674 16668 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
I20260807 15:04:37.290473 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.284013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35291 (local address 127.0.0.1:34338)
0807 15:04:37.284877 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.284910 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.285000 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.286274 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.286277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.286280 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:37.286408 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.286410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.287835 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.287836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.289020 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.289025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.290120 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.290122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.290223 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.290239 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.290322 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.290383 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":120,"threads_started":1}
I20260807 15:04:37.290490 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.284324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.1.127.193:35291)
0807 15:04:37.284511 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.284517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.285869 (+ 1352us) server_negotiation.cc:407] Connection header received
0807 15:04:37.286092 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.286097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.286117 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.286201 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:37.286493 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.287716 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.289139 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.289943 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.290027 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.290098 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.290358 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.290406 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
I20260807 15:04:37.291333 16599 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d9f96061b54e98a61d610029ac32c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:36 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:34338
I20260807 15:04:37.292133 16678 tablet_replica.cc:330] stopping tablet replica
I20260807 15:04:37.292284 16678 raft_consensus.cc:2229] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:37.292397 16678 raft_consensus.cc:2258] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:37.292743 16678 ts_tablet_manager.cc:1905] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:37.295146 16678 ts_tablet_manager.cc:1918] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:37.295259 16678 log.cc:1196] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/f9d9f96061b54e98a61d610029ac32c0
I20260807 15:04:37.295495 16678 ts_tablet_manager.cc:1939] T f9d9f96061b54e98a61d610029ac32c0 P ef05c06d2810450aa788ab5f574bd9ae: Deleting consensus metadata
I20260807 15:04:37.296706 16457 catalog_manager.cc:4841] TS ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193:35291): tablet f9d9f96061b54e98a61d610029ac32c0 (table test-table [id=248775e5660f458c97e43d51d6bca9cb]) successfully deleted
W20260807 15:04:37.372225 16672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:37.372463 16672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:37.372509 16672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:37.373760 16672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:37.373808 16672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:37.373836 16672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:37.375566 16672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:45523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=46245
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:37.375802 16672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:37.376009 16672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:37.381979 16687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.381951 16683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.381925 16685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:37.382061 16672 server_base.cc:1034] running on GCE node
I20260807 15:04:37.382447 16672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:37.382822 16672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:37.384019 16672 hybrid_clock.cc:648] HybridClock initialized: now 1786115077383995 us; error 42 us; skew 500 ppm
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:37.387224 16672 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:37.387802 16672 webserver.cc:460] Webserver started at http://127.1.127.194:46245/ using document root <none> and password file <none>
I20260807 15:04:37.388072 16672 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:37.388137 16672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:37.389734 16672 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:37.390533 16694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.390807 16672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:37.390883 16672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "7d6de7dc0fa242cd83d951f89146373b"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7cf87ddbd9d4a482928df48d28acdc36"
server_key_iv: "e2cc4113bda825ab3646f3b78bf8f8be"
server_key_version: "encryptionkey@0"
I20260807 15:04:37.391309 16672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:37.406780 16672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:37.409526 16672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:37.409714 16672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:37.410027 16672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:37.410395 16672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:37.410434 16672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.410458 16672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:37.410471 16672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.417537 16672 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45523
I20260807 15:04:37.417578 16811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45523 every 8 connection(s)
I20260807 15:04:37.418099 16672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:37.422588 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16672
I20260807 15:04:37.422926 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:38495
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=36875
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:37.430552 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.419065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:43591 (local address 127.1.127.254:41887)
0807 15:04:37.419224 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.419227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.419467 (+ 240us) server_negotiation.cc:407] Connection header received
0807 15:04:37.420166 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.420167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.420187 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.420271 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:37.421621 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.422201 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.423521 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.423744 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.423824 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.426837 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:37.426839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:37.426839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:37.426847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:37.428743 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.429344 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.429346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.429346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.429443 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.429752 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.429753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.429754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.429888 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:37.429968 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.430342 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.430450 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 15:04:37.430819 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.419049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:43591)
0807 15:04:37.419281 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.419357 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.420044 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.420355 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.420360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.421189 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:37.421513 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.421519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.422317 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.422318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.423422 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.423426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.423849 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.423850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.423987 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.424631 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:37.424645 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:37.426658 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:37.428877 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.428892 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.428895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.429247 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.429526 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.429527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.429527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.429663 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.429958 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:37.429967 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:37.430192 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.430307 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.430497 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260807 15:04:37.431388 16812 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:37.431527 16812 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:37.431758 16812 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:37.432286 16473 ts_manager.cc:194] Registered new tserver with Master: 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
I20260807 15:04:37.432703 16473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:43591
W20260807 15:04:37.518061 16815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:37.518312 16815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:37.518340 16815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:37.519855 16815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:37.520011 16815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:37.520082 16815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:37.522202 16815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:38495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=36875
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:37.522535 16815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:37.522871 16815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:04:37.529044 16815 server_base.cc:1034] running on GCE node
W20260807 15:04:37.529013 16823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.528976 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.528976 16820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:37.530059 16815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:37.530403 16815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:37.531613 16815 hybrid_clock.cc:648] HybridClock initialized: now 1786115077531594 us; error 35 us; skew 500 ppm
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:37.534446 16815 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:37.535096 16815 webserver.cc:460] Webserver started at http://127.1.127.195:36875/ using document root <none> and password file <none>
I20260807 15:04:37.535370 16815 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:37.535461 16815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:37.536980 16815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:04:37.537731 16830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.538045 16815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:37.538199 16815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "87ed77a16918afce5b5798e042d929bf"
server_key_iv: "e43a1c450dc9fe2aef8884d13a4eae2f"
server_key_version: "encryptionkey@0"
I20260807 15:04:37.538605 16815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:37.557034 16815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:37.559651 16815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:37.559835 16815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:37.560108 16815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:37.560491 16815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:37.560547 16815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.560588 16815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:37.560617 16815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.567787 16815 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38495
I20260807 15:04:37.567831 16949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38495 every 8 connection(s)
I20260807 15:04:37.568336 16815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:37.570899 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16815
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:37.580428 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.569277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:40723 (local address 127.1.127.254:41887)
0807 15:04:37.569333 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.569335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.569598 (+ 263us) server_negotiation.cc:407] Connection header received
0807 15:04:37.570189 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.570190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.570203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.570263 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:37.571528 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.572053 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.573306 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.573540 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.573627 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.576522 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:37.576524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:37.576525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:37.576533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:37.578332 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.578931 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.578933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.578933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.579060 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.579483 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.579484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.579485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.579686 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:37.579746 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.580214 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.580325 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:04:37.580685 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.569269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:40723)
0807 15:04:37.569452 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.569492 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.570066 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.570344 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.570348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.571143 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:37.571424 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.571431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.572165 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.572166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.573012 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.573015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.573656 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.573658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.573821 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.574563 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:37.574575 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:37.576399 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:37.578464 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.578474 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.578478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.578855 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.579170 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:37.579171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:37.579172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:37.579378 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:37.579785 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:37.579801 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:37.579998 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.580116 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.580341 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260807 15:04:37.580547 16954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.571438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.1.127.254:41887)
0807 15:04:37.571569 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.571572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.571675 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:04:37.571766 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.571767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.571781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.571855 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:37.572609 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.573173 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.574453 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.574937 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.575023 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.576547 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:37.576548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:37.576549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:37.576557 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:37.578599 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.579281 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.579282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.579283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.579347 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:37.579668 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:37.579670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:37.579670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:37.579852 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:37.580061 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.580224 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.580412 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260807 15:04:37.581385 16950 heartbeater.cc:347] Connected to a master server at 127.1.127.254:41887
I20260807 15:04:37.581487 16950 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:37.581656 16950 heartbeater.cc:511] Master 127.1.127.254:41887 requested a full tablet report, sending...
I20260807 15:04:37.582063 16473 ts_manager.cc:194] Registered new tserver with Master: 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495)
I20260807 15:04:37.582453 16473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:40723
I20260807 15:04:37.593766 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:04:37.597529 16954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.594331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33346 (local address 127.1.127.254:41887)
0807 15:04:37.594422 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.594424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.594515 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:04:37.594635 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.594635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.594743 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.594839 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:04:37.595075 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.595637 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.596618 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.596844 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.596908 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.597101 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:04:37.597220 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:04:37.597292 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.597362 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.597389 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:04:37.597993 16473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:04:37.599977 16473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:04:37.603202 16599 tablet_service.cc:1467] Processing CreateTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b (DEFAULT_TABLE table=test-table [id=81ae55ce5b5c47e58cdeb7d343ab270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:37.603454 16599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c6b3c47a9a4924b191fb77ab65af0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:37.605407 16556 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap starting.
I20260807 15:04:37.605805 16556 tablet_bootstrap.cc:654] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:37.606559 16556 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: No bootstrap required, opened a new log
I20260807 15:04:37.606622 16556 ts_tablet_manager.cc:1397] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:37.606781 16556 raft_consensus.cc:348] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.606859 16556 raft_consensus.cc:374] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:37.606917 16556 raft_consensus.cc:729] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Initialized, Role: FOLLOWER
I20260807 15:04:37.606992 16556 consensus_queue.cc:260] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.607096 16556 ts_tablet_manager.cc:1428] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.607657 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.602809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38495 (local address 127.0.0.1:33368)
0807 15:04:37.602959 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.602972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.603027 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.603419 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.603420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.603423 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:37.603531 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.603534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.604817 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.604819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.606005 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.606010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.607384 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.607385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.607474 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.607480 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.607513 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.607567 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 15:04:37.608021 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.602898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.1.127.195:38495)
0807 15:04:37.603204 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.603210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.603222 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:04:37.603268 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.603270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.603288 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.603385 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:37.603675 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.604683 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.606103 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.607269 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.607795 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.607889 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.607915 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.607954 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":163,"threads_started":1}
I20260807 15:04:37.608652 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.602977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45523 (local address 127.0.0.1:36820)
0807 15:04:37.603124 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.603136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.603191 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.604117 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.604119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.604121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:37.604173 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.604176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.605794 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.605795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.607216 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.607226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.608445 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.608446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.608501 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.608505 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.608559 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.608596 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260807 15:04:37.608733 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.603670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.1.127.194:45523)
0807 15:04:37.603916 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.603920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.603933 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:04:37.603977 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.603980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.603999 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.604097 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:37.604248 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.605678 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.607318 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.608271 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.608384 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.608450 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.608590 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.608636 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":120,"threads_started":1}
I20260807 15:04:37.608897 16879 tablet_service.cc:1467] Processing CreateTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b (DEFAULT_TABLE table=test-table [id=81ae55ce5b5c47e58cdeb7d343ab270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:37.609349 16879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c6b3c47a9a4924b191fb77ab65af0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:37.609592 16743 tablet_service.cc:1467] Processing CreateTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b (DEFAULT_TABLE table=test-table [id=81ae55ce5b5c47e58cdeb7d343ab270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:37.609968 16743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31c6b3c47a9a4924b191fb77ab65af0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:37.612953 16960 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Bootstrap starting.
I20260807 15:04:37.613142 16961 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Bootstrap starting.
I20260807 15:04:37.613931 16960 tablet_bootstrap.cc:654] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:37.613924 16961 tablet_bootstrap.cc:654] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:37.614708 16960 log.cc:824] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:37.614713 16961 log.cc:824] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:37.615782 16961 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: No bootstrap required, opened a new log
I20260807 15:04:37.615870 16961 ts_tablet_manager.cc:1397] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:04:37.615973 16960 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: No bootstrap required, opened a new log
I20260807 15:04:37.616042 16960 ts_tablet_manager.cc:1397] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:04:37.618021 16961 raft_consensus.cc:348] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.618090 16960 raft_consensus.cc:348] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.618245 16961 raft_consensus.cc:374] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:37.618245 16960 raft_consensus.cc:374] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:37.618287 16961 raft_consensus.cc:729] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de7dc0fa242cd83d951f89146373b, State: Initialized, Role: FOLLOWER
I20260807 15:04:37.618287 16960 raft_consensus.cc:729] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4ee237c34f47cd8e9d949b4749a8e5, State: Initialized, Role: FOLLOWER
I20260807 15:04:37.618424 16960 consensus_queue.cc:260] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.618450 16961 consensus_queue.cc:260] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.618788 16960 ts_tablet_manager.cc:1428] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:04:37.618844 16950 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
I20260807 15:04:37.618788 16961 ts_tablet_manager.cc:1428] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:04:37.618872 16812 heartbeater.cc:503] Master 127.1.127.254:41887 was elected leader, sending a full tablet report...
W20260807 15:04:37.668853 16813 tablet.cc:2369] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:04:37.770279 16669 tablet.cc:2369] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:37.778083 16674 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:37.778249 16674 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.778743 16674 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
I20260807 15:04:37.783403 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.779123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38495 (local address 127.1.127.193:43959)
0807 15:04:37.779202 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.779216 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.779270 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.779723 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.779724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.779726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:37.780070 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.780073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.780891 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.780892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.782141 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.782143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.783164 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.783164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.783231 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.783234 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.783287 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.783326 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 15:04:37.783454 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.779289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:43959 (local address 127.1.127.195:38495)
0807 15:04:37.779369 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.779371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.779384 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:04:37.779422 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.779423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.779434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.779551 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:37.780169 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.780815 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.782252 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.783076 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.783156 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.783163 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.783331 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.783385 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:04:37.783412 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.779279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45523 (local address 127.1.127.193:39931)
0807 15:04:37.779383 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.779392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:37.779440 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:37.779681 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:37.779682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:37.779684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:37.779746 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.779749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.780699 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.780701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.782079 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:37.782083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.783089 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.783091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:37.783182 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.783187 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:04:37.783256 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:04:37.783309 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":4,"thread_start_us":71,"threads_started":1}
I20260807 15:04:37.783501 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.779475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:39931 (local address 127.1.127.194:45523)
0807 15:04:37.779555 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.779557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.779571 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:04:37.779596 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.779597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.779604 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.779673 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:37.779857 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.780569 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.782269 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.782968 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.783065 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.783075 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.783319 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.783428 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:04:37.783914 16900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c6b3c47a9a4924b191fb77ab65af0b" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" is_pre_election: true
I20260807 15:04:37.783914 16764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c6b3c47a9a4924b191fb77ab65af0b" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de7dc0fa242cd83d951f89146373b" is_pre_election: true
I20260807 15:04:37.784093 16900 raft_consensus.cc:2454] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 0.
I20260807 15:04:37.784093 16764 raft_consensus.cc:2454] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 0.
I20260807 15:04:37.784394 16550 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de7dc0fa242cd83d951f89146373b, ef05c06d2810450aa788ab5f574bd9ae; no voters:
I20260807 15:04:37.784554 16674 raft_consensus.cc:2781] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:04:37.784627 16674 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:04:37.784657 16674 raft_consensus.cc:3037] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:37.785459 16674 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.785625 16674 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 election: Requested vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
I20260807 15:04:37.785838 16900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c6b3c47a9a4924b191fb77ab65af0b" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
I20260807 15:04:37.785885 16764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31c6b3c47a9a4924b191fb77ab65af0b" candidate_uuid: "ef05c06d2810450aa788ab5f574bd9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de7dc0fa242cd83d951f89146373b"
I20260807 15:04:37.785930 16900 raft_consensus.cc:3037] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:37.786006 16764 raft_consensus.cc:3037] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:37.786800 16900 raft_consensus.cc:2454] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 1.
I20260807 15:04:37.786963 16764 raft_consensus.cc:2454] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef05c06d2810450aa788ab5f574bd9ae in term 1.
I20260807 15:04:37.787001 16550 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d4ee237c34f47cd8e9d949b4749a8e5, ef05c06d2810450aa788ab5f574bd9ae; no voters:
I20260807 15:04:37.787195 16674 raft_consensus.cc:2781] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:37.787417 16674 raft_consensus.cc:686] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 LEADER]: Becoming Leader. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Running, Role: LEADER
I20260807 15:04:37.787515 16674 consensus_queue.cc:237] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:37.788441 16472 catalog_manager.cc:5495] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae reported cstate change: term changed from 0 to 1, leader changed from <none> to ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193). New cstate: current_term: 1 leader_uuid: "ef05c06d2810450aa788ab5f574bd9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } health_report { overall_health: HEALTHY } } }
W20260807 15:04:37.819046 16951 tablet.cc:2369] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:37.838006 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.835152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.1.127.193:35291)
0807 15:04:37.835291 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:37.835293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:37.835301 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:04:37.835344 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:37.835345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:37.835366 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:37.835444 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:04:37.835564 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.836106 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.836940 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:37.837244 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:37.837312 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:37.837588 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:04:37.837705 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:04:37.837781 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:37.837848 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:04:37.837872 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260807 15:04:37.839962 16900 raft_consensus.cc:1261] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Refusing update from remote peer ef05c06d2810450aa788ab5f574bd9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:37.839973 16764 raft_consensus.cc:1261] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Refusing update from remote peer ef05c06d2810450aa788ab5f574bd9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:37.840334 16967 consensus_queue.cc:1035] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:37.840490 16674 consensus_queue.cc:1035] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:37.843340 16970 mvcc.cc:205] Tried to move back new op lower bound from 7315927358829223936 to 7315927358618484736. Current Snapshot: MvccSnapshot[applied={T|T < 7315927358829223936}]
I20260807 15:04:37.843873 16971 mvcc.cc:205] Tried to move back new op lower bound from 7315927358829223936 to 7315927358618484736. Current Snapshot: MvccSnapshot[applied={T|T < 7315927358829223936}]
I20260807 15:04:37.844100 16972 mvcc.cc:205] Tried to move back new op lower bound from 7315927358829223936 to 7315927358618484736. Current Snapshot: MvccSnapshot[applied={T|T < 7315927358829223936}]
I20260807 15:04:37.845794 16473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:37.845888 16473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:37.846742 16473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Sending DeleteTablet for 3 replicas of tablet 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:37.847157 16599 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:37 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:34338
I20260807 15:04:37.847223 16879 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:37 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:33368
I20260807 15:04:37.847290 16743 tablet_service.cc:1514] Processing DeleteTablet for tablet 31c6b3c47a9a4924b191fb77ab65af0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:37 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:36820
I20260807 15:04:37.847512 16981 tablet_replica.cc:330] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: stopping tablet replica
I20260807 15:04:37.847584 16981 raft_consensus.cc:2229] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:04:37.847632 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16513
I20260807 15:04:37.847702 16981 raft_consensus.cc:2258] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:37.849385 16982 tablet_replica.cc:330] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: stopping tablet replica
I20260807 15:04:37.849570 16982 raft_consensus.cc:2229] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:37.849722 16982 raft_consensus.cc:2258] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:37.850265 16982 ts_tablet_manager.cc:1905] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:37.851979 16980 tablet_replica.cc:330] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: stopping tablet replica
I20260807 15:04:37.852166 16980 raft_consensus.cc:2229] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:37.852360 16980 raft_consensus.cc:2258] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:37.852512 16982 ts_tablet_manager.cc:1918] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:37.852627 16982 log.cc:1196] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:37.852948 16980 ts_tablet_manager.cc:1905] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:37.852988 16982 ts_tablet_manager.cc:1939] T 31c6b3c47a9a4924b191fb77ab65af0b P 7d6de7dc0fa242cd83d951f89146373b: Deleting consensus metadata
I20260807 15:04:37.853760 16456 catalog_manager.cc:4841] TS 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): tablet 31c6b3c47a9a4924b191fb77ab65af0b (table test-table [id=81ae55ce5b5c47e58cdeb7d343ab270f]) successfully deleted
I20260807 15:04:37.855230 16980 ts_tablet_manager.cc:1918] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:37.855340 16980 log.cc:1196] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:37.855726 16980 ts_tablet_manager.cc:1939] T 31c6b3c47a9a4924b191fb77ab65af0b P 6d4ee237c34f47cd8e9d949b4749a8e5: Deleting consensus metadata
I20260807 15:04:37.856287 16456 catalog_manager.cc:4841] TS 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): tablet 31c6b3c47a9a4924b191fb77ab65af0b (table test-table [id=81ae55ce5b5c47e58cdeb7d343ab270f]) successfully deleted
W20260807 15:04:37.858160 16457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:35291 (error 108)
I20260807 15:04:37.858368 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16672
I20260807 15:04:37.860894 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:37.860709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35291 (local address 127.0.0.1:34360)
0807 15:04:37.860784 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:37.860842 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:04:37.861101 16457 catalog_manager.cc:4584] TS ef05c06d2810450aa788ab5f574bd9ae (127.1.127.193:35291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31c6b3c47a9a4924b191fb77ab65af0b: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35291: connect: Connection refused (error 111)
I20260807 15:04:37.865464 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16815
I20260807 15:04:37.871625 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 16437
I20260807 15:04:37.879377 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--webserver_interface=127.1.127.254
--webserver_port=35355
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:37.965073 16984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:37.965296 16984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:37.965320 16984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:37.966568 16984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:37.966621 16984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:37.966635 16984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:37.966650 16984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:37.966661 16984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:37.968360 16984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=35355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:37.968638 16984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:37.968946 16984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:37.974732 16989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.974695 16990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:37.974995 16992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:37.975023 16984 server_base.cc:1034] running on GCE node
I20260807 15:04:37.975296 16984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:37.975605 16984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:37.976778 16984 hybrid_clock.cc:648] HybridClock initialized: now 1786115077976758 us; error 43 us; skew 500 ppm
Aug 07 15:04:37 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115077, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:37.979987 16984 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:37.980551 16984 webserver.cc:460] Webserver started at http://127.1.127.254:35355/ using document root <none> and password file <none>
I20260807 15:04:37.980813 16984 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:37.980899 16984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:37.982401 16984 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:37.983373 16999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:37.983673 16984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:37.983826 16984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "f1d8c43900c64dba9a93c416744e6aa0"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "13fba8c8b4973b9f7c79763b9e1efb68"
server_key_iv: "4394797a2ae85c9f974787808f3b2c07"
server_key_version: "encryptionkey@0"
I20260807 15:04:37.984341 16984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:38.001508 16984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:38.004390 16984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:38.004628 16984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:38.010576 16984 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41887
I20260807 15:04:38.010612 17053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41887 every 8 connection(s)
I20260807 15:04:38.011144 16984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:38.012950 17054 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:04:38.013700 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap starting.
I20260807 15:04:38.015672 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 16984
I20260807 15:04:38.016042 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:35291
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=40845
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:04:38.024518 17054 log.cc:824] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:38.027029 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:04:38.027396 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0: Bootstrap complete.
I20260807 15:04:38.030061 17054 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:38.030304 17054 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Initialized, Role: FOLLOWER
I20260807 15:04:38.030491 17054 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:38.030556 17054 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:38.030624 17054 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:38.030704 17054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:04:38.031806 17054 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:38.031911 17054 leader_election.cc:302] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d8c43900c64dba9a93c416744e6aa0; no voters:
I20260807 15:04:38.032140 17054 leader_election.cc:288] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:04:38.032183 17058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:04:38.032294 17058 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [term 3 LEADER]: Becoming Leader. State: Replica: f1d8c43900c64dba9a93c416744e6aa0, State: Running, Role: LEADER
I20260807 15:04:38.032449 17054 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:38.032474 17058 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } }
I20260807 15:04:38.032776 17058 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d8c43900c64dba9a93c416744e6aa0. Latest consensus state: current_term: 3 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:38.032862 17058 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:38.032784 17060 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f1d8c43900c64dba9a93c416744e6aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8c43900c64dba9a93c416744e6aa0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41887 } } }
I20260807 15:04:38.033041 17060 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8c43900c64dba9a93c416744e6aa0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:38.033084 17064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:38.034130 17064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:38.034483 17064 catalog_manager.cc:1259] Loaded cluster ID: d15fb114a9aa4ebc96fa7c1f312b2004
I20260807 15:04:38.034536 17064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 15:04:38.035053 17064 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f718adf79ad google::LogMessage::Fail() at ??:0
@ 0x7f718adfa88a google::LogMessage::SendToLog() at ??:0
@ 0x7f718adf74de google::LogMessage::Flush() at ??:0
@ 0x7f718adf8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7192d180f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7192d218b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f718ccc50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f718ccba954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f718db1e6db start_thread at ??:0
@ 0x7f71898f561f clone at ??:0
W20260807 15:04:38.121162 17056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:38.121397 17056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:38.121433 17056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:38.122810 17056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:38.122861 17056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:38.122892 17056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:38.124565 17056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:35291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=40845
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:38.124843 17056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:38.125124 17056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:38.130846 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:38.130857 17091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:38.131081 17056 server_base.cc:1034] running on GCE node
W20260807 15:04:38.130857 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:38.131510 17056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:38.131829 17056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:38.133038 17056 hybrid_clock.cc:648] HybridClock initialized: now 1786115078133036 us; error 53 us; skew 500 ppm
Aug 07 15:04:38 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:38.136193 17056 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:38.136806 17056 webserver.cc:460] Webserver started at http://127.1.127.193:40845/ using document root <none> and password file <none>
I20260807 15:04:38.137111 17056 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:38.137189 17056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:38.139285 17056 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:04:38.140193 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.140451 17056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:38.140551 17056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "ef05c06d2810450aa788ab5f574bd9ae"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7db86ffdca5ed63cb0e4446e0b502302"
server_key_iv: "16f57ba515a009a90a2f3681b7129bcc"
server_key_version: "encryptionkey@0"
I20260807 15:04:38.141041 17056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:38.158964 17056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:38.161747 17056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:38.161921 17056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:38.162222 17056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:38.162813 17109 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:04:38.164496 17056 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:04:38.164567 17056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:04:38.164605 17056 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:04:38.165575 17056 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:04:38.165624 17056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:38.165717 17109 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap starting.
I20260807 15:04:38.173249 17056 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35291
I20260807 15:04:38.173698 17220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35291 every 8 connection(s)
I20260807 15:04:38.173935 17056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:38.177707 17109 log.cc:824] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:38.180143 17109 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:04:38.180474 17109 tablet_bootstrap.cc:492] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Bootstrap complete.
I20260807 15:04:38.181510 17109 ts_tablet_manager.cc:1397] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260807 15:04:38.182611 17109 raft_consensus.cc:348] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:38.182967 17109 raft_consensus.cc:729] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef05c06d2810450aa788ab5f574bd9ae, State: Initialized, Role: FOLLOWER
I20260807 15:04:38.183156 17109 consensus_queue.cc:260] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:38.183512 17109 ts_tablet_manager.cc:1428] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20260807 15:04:38.183526 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17056
I20260807 15:04:38.183770 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:45523
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=46245
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:38.244455 17223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:04:38.174903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:40777)
0807 15:04:38.175173 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.175240 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:38.176036 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:38.244154 (+ 68118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":121,"threads_started":1}
W20260807 15:04:38.244730 17221 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41887 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:41887: Client connection negotiation failed: client connection to 127.1.127.254:41887: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:04:38.245116 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.244978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:50853)
0807 15:04:38.245041 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.245095 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:04:38.245564 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.245474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:47375)
0807 15:04:38.245522 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.245548 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:04:38.245730 17221 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:04:38.273267 17226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:38.273512 17226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:38.273540 17226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:38.274852 17226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:38.274901 17226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:38.274930 17226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:38.276497 17226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:45523
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=46245
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:38.276765 17226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:38.277011 17226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:38.282343 17231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:38.282346 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:38.282343 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:38.282889 17226 server_base.cc:1034] running on GCE node
I20260807 15:04:38.283141 17226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:38.283417 17226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:38.284607 17226 hybrid_clock.cc:648] HybridClock initialized: now 1786115078284572 us; error 55 us; skew 500 ppm
Aug 07 15:04:38 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:38.287446 17226 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:38.287992 17226 webserver.cc:460] Webserver started at http://127.1.127.194:46245/ using document root <none> and password file <none>
I20260807 15:04:38.288249 17226 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:38.288297 17226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:38.289814 17226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:38.290637 17241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.290894 17226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:38.291074 17226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "7d6de7dc0fa242cd83d951f89146373b"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "7cf87ddbd9d4a482928df48d28acdc36"
server_key_iv: "e2cc4113bda825ab3646f3b78bf8f8be"
server_key_version: "encryptionkey@0"
I20260807 15:04:38.291493 17226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:38.311002 17226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:38.313488 17226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:38.313689 17226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:38.313970 17226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:38.314317 17226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:38.314379 17226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.314419 17226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:38.314450 17226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.321435 17226 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45523
I20260807 15:04:38.321489 17359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45523 every 8 connection(s)
I20260807 15:04:38.321913 17226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:38.323243 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.322740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:37661)
0807 15:04:38.322992 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.323088 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
W20260807 15:04:38.323760 17360 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41887 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:41887: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
I20260807 15:04:38.324098 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.323980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:54887)
0807 15:04:38.324036 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.324064 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:04:38.324509 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.324412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:39275)
0807 15:04:38.324461 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.324486 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 15:04:38.324638 17360 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:04:38.330588 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17226
I20260807 15:04:38.330972 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:38495
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:41887
--webserver_port=36875
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:38.419230 17363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:38.419473 17363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:38.419500 17363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:38.420760 17363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:38.420825 17363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:38.420861 17363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:38.422431 17363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:38495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=36875
--tserver_master_addrs=127.1.127.254:41887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:38.422662 17363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:38.422955 17363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:38.429068 17370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:38.429137 17363 server_base.cc:1034] running on GCE node
W20260807 15:04:38.429061 17372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:38.429067 17369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:38.429708 17363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:38.430078 17363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:38.431252 17363 hybrid_clock.cc:648] HybridClock initialized: now 1786115078431227 us; error 47 us; skew 500 ppm
Aug 07 15:04:38 dist-test-slave-8tdl krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:38.434275 17363 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:38.435000 17363 webserver.cc:460] Webserver started at http://127.1.127.195:36875/ using document root <none> and password file <none>
I20260807 15:04:38.435271 17363 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:38.435325 17363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:38.436921 17363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:04:38.437709 17379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.438043 17363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:38.438174 17363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "6d4ee237c34f47cd8e9d949b4749a8e5"
format_stamp: "Formatted at 2026-08-07 15:04:36 on dist-test-slave-8tdl"
server_key: "87ed77a16918afce5b5798e042d929bf"
server_key_iv: "e43a1c450dc9fe2aef8884d13a4eae2f"
server_key_version: "encryptionkey@0"
I20260807 15:04:38.438584 17363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:38.462532 17363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:38.464996 17363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:38.465196 17363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:38.465476 17363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:38.465891 17363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:38.466030 17363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.466081 17363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:38.466109 17363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:38.472615 17363 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38495
I20260807 15:04:38.472674 17498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38495 every 8 connection(s)
I20260807 15:04:38.473052 17363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:38.474289 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.473816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:49917)
0807 15:04:38.474034 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.474121 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
W20260807 15:04:38.474852 17499 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:41887 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:41887: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
I20260807 15:04:38.475270 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.475158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:42383)
0807 15:04:38.475213 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.475246 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:04:38.475647 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:38.475560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:44113)
0807 15:04:38.475604 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:38.475629 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:04:38.475780 17499 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:04:38.477941 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17363
I20260807 15:04:39.246500 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.246173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:57145)
0807 15:04:39.246383 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:39.246432 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260807 15:04:39.325327 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.325019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:42607)
0807 15:04:39.325214 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:39.325260 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260807 15:04:39.476424 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.476147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:51011)
0807 15:04:39.476326 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:39.476364 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260807 15:04:39.736779 17506 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:39.736912 17506 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:39.737373 17506 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
Aug 07 15:04:39 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.195@KRBTEST.COM
Aug 07 15:04:39 dist-test-slave-8tdl krb5kdc[15878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.194@KRBTEST.COM
I20260807 15:04:39.749876 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.737738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38495 (local address 127.1.127.193:58203)
0807 15:04:39.737827 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:39.737845 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:39.737933 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:39.739031 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:39.739034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:39.739507 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:39.739706 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:39.739713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.740994 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.740995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:39.742147 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:39.742150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.742900 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:39.743026 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:39.743729 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:39.743733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:39.745454 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:39.748030 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:39.748039 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:39.748042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:39.748285 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:39.748579 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:39.748580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:39.748580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:39.748696 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:39.749314 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:39.749322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:39.749465 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 15:04:39.749579 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:04:39.749766 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":3}
I20260807 15:04:39.749889 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.737883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:58203 (local address 127.1.127.195:38495)
0807 15:04:39.738149 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:39.738163 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:39.738200 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:04:39.738766 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:39.738769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:39.738819 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:39.738956 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:39.739984 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.740885 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742257 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.742799 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742930 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:39.745654 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:39.745660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:39.745660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:39.745698 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:39.747908 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:39.748380 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:39.748384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:39.748384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:39.748493 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:39.748775 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:39.748775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:39.748776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:39.749195 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:39.749302 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:39.749636 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 15:04:39.749765 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":159,"threads_started":1}
I20260807 15:04:39.750298 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.737939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45523 (local address 127.1.127.193:33553)
0807 15:04:39.738086 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:39.738098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:39.738148 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:39.739097 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:39.739098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:39.739490 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:39.739706 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:39.739713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.741221 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.741222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:39.742123 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:39.742126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.742903 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:39.743028 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:39.743661 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:39.743670 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:39.746777 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:39.748849 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:39.748850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:39.748850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:39.749048 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:39.749378 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:39.749378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:39.749379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:39.749453 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:39.749886 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:39.749886 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:39.749932 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:04:39.750007 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:04:39.750126 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":5,"thread_start_us":86,"threads_started":1}
I20260807 15:04:39.750360 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:39.738100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:33553 (local address 127.1.127.194:45523)
0807 15:04:39.738259 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:39.738266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:39.738294 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:04:39.738818 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:39.738821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:39.738862 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:39.739107 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:39.739892 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.740989 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742209 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:39.742799 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:39.742934 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:39.746931 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:39.746937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:39.746937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:39.746951 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:39.748772 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:39.749157 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:39.749160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:39.749161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:39.749266 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:39.749565 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:39.749566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:39.749567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:39.749808 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:39.749903 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:39.750090 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:04:39.750229 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
W20260807 15:04:39.751195 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
W20260807 15:04:39.751303 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:39.751338 17103 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef05c06d2810450aa788ab5f574bd9ae; no voters: 6d4ee237c34f47cd8e9d949b4749a8e5, 7d6de7dc0fa242cd83d951f89146373b
I20260807 15:04:39.751503 17506 raft_consensus.cc:2726] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:04:40.247400 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:40.247162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:46085)
0807 15:04:40.247250 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:40.247301 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:04:40.326351 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:40.326094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:33187)
0807 15:04:40.326255 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:40.326303 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260807 15:04:40.477450 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:40.477133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:41457)
0807 15:04:40.477302 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:40.477396 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260807 15:04:41.248282 17512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:41.248032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:43147)
0807 15:04:41.248187 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:41.248234 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260807 15:04:41.327193 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:41.326929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:54437)
0807 15:04:41.327106 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:41.327144 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260807 15:04:41.478430 17514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:41.478131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:53399)
0807 15:04:41.478329 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:41.478378 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260807 15:04:41.937827 17515 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:41.937973 17515 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:41.938201 17515 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
W20260807 15:04:41.938736 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
W20260807 15:04:41.938807 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:41.938825 17103 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef05c06d2810450aa788ab5f574bd9ae; no voters: 6d4ee237c34f47cd8e9d949b4749a8e5, 7d6de7dc0fa242cd83d951f89146373b
I20260807 15:04:41.938896 17515 raft_consensus.cc:2726] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:04:42.249305 17516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:42.249045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:46423)
0807 15:04:42.249208 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:42.249253 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260807 15:04:42.328090 17517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:42.327769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:58031)
0807 15:04:42.327903 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:42.327960 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260807 15:04:42.479349 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:42.479076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:59405)
0807 15:04:42.479247 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:42.479297 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260807 15:04:43.250392 17519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:43.250118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:43145)
0807 15:04:43.250291 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:43.250340 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260807 15:04:43.329058 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:43.328729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:41277)
0807 15:04:43.328889 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:43.328999 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260807 15:04:43.480352 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:43.480094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:56837)
0807 15:04:43.480249 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:43.480299 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260807 15:04:44.236630 17522 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:44.236779 17522 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:44.237007 17522 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
W20260807 15:04:44.237457 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
W20260807 15:04:44.237568 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:44.237607 17103 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef05c06d2810450aa788ab5f574bd9ae; no voters: 6d4ee237c34f47cd8e9d949b4749a8e5, 7d6de7dc0fa242cd83d951f89146373b
I20260807 15:04:44.237700 17522 raft_consensus.cc:2726] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:04:44.251354 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:44.251085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:46571)
0807 15:04:44.251246 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:44.251299 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260807 15:04:44.329916 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:44.329669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:56555)
0807 15:04:44.329818 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:44.329862 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260807 15:04:44.481257 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:44.481006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:47467)
0807 15:04:44.481158 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:44.481204 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260807 15:04:45.252377 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:45.252101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:52901)
0807 15:04:45.252267 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:45.252319 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260807 15:04:45.330905 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:45.330580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:45467)
0807 15:04:45.330802 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:45.330851 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":74,"threads_started":1}
I20260807 15:04:45.482204 17529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:45.481907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:59579)
0807 15:04:45.482102 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:45.482152 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260807 15:04:46.253170 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:46.252851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:55619)
0807 15:04:46.253050 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:46.253102 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":63,"threads_started":1}
I20260807 15:04:46.331736 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:46.331478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:44583)
0807 15:04:46.331641 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:46.331686 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260807 15:04:46.483115 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:46.482861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:41813)
0807 15:04:46.483011 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:46.483064 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260807 15:04:47.254099 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:47.253828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:33831)
0807 15:04:47.253991 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:47.254049 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260807 15:04:47.332756 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:47.332438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:52569)
0807 15:04:47.332636 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:47.332673 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260807 15:04:47.444084 17535 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:47.444322 17535 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:47.444682 17535 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
W20260807 15:04:47.445132 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
W20260807 15:04:47.445290 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:47.445338 17103 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef05c06d2810450aa788ab5f574bd9ae; no voters: 6d4ee237c34f47cd8e9d949b4749a8e5, 7d6de7dc0fa242cd83d951f89146373b
I20260807 15:04:47.445421 17535 raft_consensus.cc:2726] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:04:47.483911 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:47.483655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:35239)
0807 15:04:47.483817 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:47.483857 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":65,"threads_started":1}
I20260807 15:04:48.255225 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:48.254865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:50315)
0807 15:04:48.255109 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:48.255149 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":196,"threads_started":1}
I20260807 15:04:48.333774 17538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:48.333474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:40607)
0807 15:04:48.333673 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:48.333720 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260807 15:04:48.484938 17539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:48.484567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:34469)
0807 15:04:48.484751 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:48.484798 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260807 15:04:49.256341 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:49.255828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:34095)
0807 15:04:49.256015 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:49.256282 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260807 15:04:49.334832 17541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:49.334497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:48405)
0807 15:04:49.334648 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:49.334768 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260807 15:04:49.485896 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:49.485626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:46369)
0807 15:04:49.485796 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:49.485846 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260807 15:04:50.257292 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:50.256957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:52371)
0807 15:04:50.257187 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:50.257240 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260807 15:04:50.335712 17544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:50.335478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:46355)
0807 15:04:50.335620 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:50.335659 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260807 15:04:50.487025 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:50.486654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:54213)
0807 15:04:50.486840 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:50.486892 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260807 15:04:51.107076 17546 raft_consensus.cc:482] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:51.107182 17546 raft_consensus.cc:504] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d4ee237c34f47cd8e9d949b4749a8e5" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38495 } } peers { permanent_uuid: "7d6de7dc0fa242cd83d951f89146373b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45523 } } peers { permanent_uuid: "ef05c06d2810450aa788ab5f574bd9ae" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35291 } }
I20260807 15:04:51.107416 17546 leader_election.cc:288] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495), 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523)
W20260807 15:04:51.107929 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d4ee237c34f47cd8e9d949b4749a8e5 (127.1.127.195:38495): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
W20260807 15:04:51.108029 17103 leader_election.cc:341] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7d6de7dc0fa242cd83d951f89146373b (127.1.127.194:45523): Not found: Tablet not found: 31c6b3c47a9a4924b191fb77ab65af0b
I20260807 15:04:51.108090 17103 leader_election.cc:302] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef05c06d2810450aa788ab5f574bd9ae; no voters: 6d4ee237c34f47cd8e9d949b4749a8e5, 7d6de7dc0fa242cd83d951f89146373b
I20260807 15:04:51.108167 17546 raft_consensus.cc:2726] T 31c6b3c47a9a4924b191fb77ab65af0b P ef05c06d2810450aa788ab5f574bd9ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:04:51.258208 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:51.257920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:58551)
0807 15:04:51.258102 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:51.258152 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260807 15:04:51.336767 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:51.336447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:33265)
0807 15:04:51.336618 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:51.336696 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260807 15:04:51.488060 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:51.487783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:57921)
0807 15:04:51.487953 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:51.488007 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260807 15:04:52.259229 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:52.258971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:58051)
0807 15:04:52.259133 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:52.259180 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260807 15:04:52.337997 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:52.337548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:43345)
0807 15:04:52.337727 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:52.337937 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
I20260807 15:04:52.488945 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:52.488624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:45171)
0807 15:04:52.488792 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:52.488835 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260807 15:04:53.260114 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:53.259857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.193:52801)
0807 15:04:53.260010 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:53.260058 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260807 15:04:53.338997 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:53.338672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.194:42339)
0807 15:04:53.338893 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:53.338945 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260807 15:04:53.483899 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17056
I20260807 15:04:53.489909 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:53.489673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:41887 (local address 127.1.127.195:43951)
0807 15:04:53.489825 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:53.489862 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:41887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260807 15:04:53.491072 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17226
I20260807 15:04:53.496397 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17363
2026-08-07T15:04:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18689 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:04:53 dist-test-slave-8tdl krb5kdc[17558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:04:53 dist-test-slave-8tdl krb5kdc[17558](info): set up 2 sockets
Aug 07 15:04:53 dist-test-slave-8tdl krb5kdc[17558](info): commencing operation
krb5kdc: starting...
W20260807 15:04:54.622591 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:04:54 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:04:54.775141 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:54.869279 17572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:54.869529 17572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:54.869554 17572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:54.870813 17572 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:04:54.870854 17572 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:04:54.870867 17572 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:04:54.870879 17572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:04:54.870893 17572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:54.870906 17572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:54.870919 17572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:04:54.870931 17572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:04:54.872555 17572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:40173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:54.872772 17572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:54.872972 17572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:04:54.879135 17572 server_base.cc:1034] running on GCE node
W20260807 15:04:54.879209 17577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:54.879136 17580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:54.879256 17578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:54.879812 17572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:54.880124 17572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:54.881301 17572 hybrid_clock.cc:648] HybridClock initialized: now 1786115094881289 us; error 36 us; skew 500 ppm
Aug 07 15:04:54 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115094, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:54.884292 17572 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:04:54.884851 17572 webserver.cc:460] Webserver started at http://127.1.127.254:45631/ using document root <none> and password file <none>
I20260807 15:04:54.885078 17572 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:54.885120 17572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:54.885218 17572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:54.886292 17572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee"
format_stamp: "Formatted at 2026-08-07 15:04:54 on dist-test-slave-8tdl"
server_key: "499fc365cfc50a9b11b054106860a411"
server_key_iv: "2f7f7781e3be57d755cf6e2360eaa888"
server_key_version: "encryptionkey@0"
I20260807 15:04:54.886626 17572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee"
format_stamp: "Formatted at 2026-08-07 15:04:54 on dist-test-slave-8tdl"
server_key: "499fc365cfc50a9b11b054106860a411"
server_key_iv: "2f7f7781e3be57d755cf6e2360eaa888"
server_key_version: "encryptionkey@0"
I20260807 15:04:54.888177 17572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:54.889235 17587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:54.889500 17572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:54.889607 17572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee"
format_stamp: "Formatted at 2026-08-07 15:04:54 on dist-test-slave-8tdl"
server_key: "499fc365cfc50a9b11b054106860a411"
server_key_iv: "2f7f7781e3be57d755cf6e2360eaa888"
server_key_version: "encryptionkey@0"
I20260807 15:04:54.889673 17572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:54.909663 17572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:54.912685 17572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:54.912822 17572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:54.917891 17572 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:40173
I20260807 15:04:54.917974 17641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:40173 every 8 connection(s)
I20260807 15:04:54.918411 17572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:04:54.919494 17642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:54.922358 17642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap starting.
I20260807 15:04:54.923162 17642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:54.923542 17642 log.cc:824] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:54.924482 17642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: No bootstrap required, opened a new log
I20260807 15:04:54.925395 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17572
I20260807 15:04:54.925514 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:04:54.925707 1535 external_mini_cluster.cc:1428] Setting key 63b5e94fe5ef20b13b9a7e3a424a8e3b
I20260807 15:04:54.926880 17642 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:04:54.927150 17642 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:54.927220 17642 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Initialized, Role: FOLLOWER
I20260807 15:04:54.927404 17642 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:04:54.927505 17642 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:04:54.927551 17642 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:04:54.927603 17642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:54.928253 17642 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:04:54.928381 17642 leader_election.cc:302] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab3d11a7d8184879bfaacc2a5e7c9cee; no voters:
I20260807 15:04:54.928599 17642 leader_election.cc:288] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:04:54.928642 17647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:54.928763 17647 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 LEADER]: Becoming Leader. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Running, Role: LEADER
I20260807 15:04:54.928895 17642 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:04:54.928911 17647 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:04:54.929329 17647 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab3d11a7d8184879bfaacc2a5e7c9cee. Latest consensus state: current_term: 1 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:04:54.929627 17647 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:54.929538 17648 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:04:54.930137 17648 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:04:54.930393 17656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:04:54.931154 17656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:04:54.932967 17656 catalog_manager.cc:1347] Generated new cluster ID: a8cdb8c5dd2e4a53ba78c94aeebf8035
I20260807 15:04:54.933025 17656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:04:54 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:54.938495 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:54.926311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.1.127.254:40173)
0807 15:04:54.926568 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:54.926585 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:54.926614 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:04:54.926841 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:54.926851 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:54.926909 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:54.927057 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:54.927837 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:54.929447 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:54.930563 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:54.931205 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:54.931372 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:54.934484 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:54.934489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:54.934489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:54.934503 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:54.936673 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:54.937108 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:54.937112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:54.937112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:54.937247 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:54.937466 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:54.937467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:54.937467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:54.937808 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:54.937984 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:54.938063 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 15:04:54.938220 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":125,"threads_started":1}
I20260807 15:04:54.953094 17656 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:04:54.953915 17656 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:04:54.962021 17656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Generated new TSK 0
I20260807 15:04:54.962270 17656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:04:54.999473 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:55.094020 17669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:55.094306 17669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:55.094338 17669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:55.095762 17669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:55.095820 17669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:55.095858 17669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:04:55.097550 17669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:55.097862 17669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:55.098126 17669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:55.104127 17675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.104122 17674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.104122 17677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:55.104615 17669 server_base.cc:1034] running on GCE node
I20260807 15:04:55.104818 17669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:55.105144 17669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:55.106320 17669 hybrid_clock.cc:648] HybridClock initialized: now 1786115095106295 us; error 31 us; skew 500 ppm
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:55.109143 17669 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:04:55.109588 17669 webserver.cc:460] Webserver started at http://127.1.127.193:42097/ using document root <none> and password file <none>
I20260807 15:04:55.109854 17669 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:55.109927 17669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:55.110092 17669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:55.111322 17669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "e0373ae1509a486d80e6026dff5f83b6"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "4c6736413e6f3b762450bbe3fdf14803"
server_key_iv: "7a9a06b4df13afb31d120a6ac28e0491"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.111768 17669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "e0373ae1509a486d80e6026dff5f83b6"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "4c6736413e6f3b762450bbe3fdf14803"
server_key_iv: "7a9a06b4df13afb31d120a6ac28e0491"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.113262 17669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:04:55.114303 17684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.114554 17669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:04:55.114714 17669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "e0373ae1509a486d80e6026dff5f83b6"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "4c6736413e6f3b762450bbe3fdf14803"
server_key_iv: "7a9a06b4df13afb31d120a6ac28e0491"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.114818 17669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:55.132910 17669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:55.135890 17669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:55.136291 17669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:55.136653 17669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:55.137149 17669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:55.137219 17669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.137261 17669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:55.137308 17669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.144348 17669 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35843
I20260807 15:04:55.144421 17800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35843 every 8 connection(s)
I20260807 15:04:55.144830 17669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:04:55.147145 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17669
I20260807 15:04:55.147238 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:04:55.147477 1535 external_mini_cluster.cc:1428] Setting key 664d1c6b1445115c0e7a91c9d7db6229
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:55.157006 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.145736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:36737 (local address 127.1.127.254:40173)
0807 15:04:55.145831 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.145833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.146088 (+ 255us) server_negotiation.cc:407] Connection header received
0807 15:04:55.146657 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.146658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.146737 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.146811 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:55.148578 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.149153 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.150520 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.150817 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.150897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.153800 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:55.153802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:55.153802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:55.153813 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:55.155440 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.155911 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.155912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.155912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.155984 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.156253 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.156254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.156255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.156385 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:55.156458 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.156775 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.156894 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:04:55.157183 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.145648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:36737)
0807 15:04:55.145836 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.145953 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.146517 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.146902 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.146906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.147813 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:55.148467 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.148475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.149269 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.149271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.150389 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.150394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.150929 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.150930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.151130 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.151759 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:55.151771 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:55.153682 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:55.155555 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.155566 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.155569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.155826 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.156065 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.156065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.156066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.156186 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.156468 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:55.156477 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:55.156633 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.156751 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.156926 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
I20260807 15:04:55.157927 17801 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:04:55.158061 17801 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:55.158322 17801 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:04:55.159067 17604 ts_manager.cc:194] Registered new tserver with Master: e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843)
I20260807 15:04:55.159780 17604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:36737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:04:55.183609 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:55.277081 17806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:55.277323 17806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:55.277350 17806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:55.278729 17806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:55.278800 17806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:55.278832 17806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:04:55.280476 17806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:55.280777 17806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:55.281056 17806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:55.286566 17813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.286549 17815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.286542 17812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:55.286893 17806 server_base.cc:1034] running on GCE node
I20260807 15:04:55.287195 17806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:55.287472 17806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:55.288668 17806 hybrid_clock.cc:648] HybridClock initialized: now 1786115095288641 us; error 42 us; skew 500 ppm
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:55.291981 17806 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:04:55.292605 17806 webserver.cc:460] Webserver started at http://127.1.127.194:42641/ using document root <none> and password file <none>
I20260807 15:04:55.292871 17806 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:55.292946 17806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:55.293097 17806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:55.294267 17806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "840507dc2d394573958948e981180243"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "bd2c8663c7b0b014109b86cd3230c3ee"
server_key_iv: "c210e9cdd5be6e426ef8fd568403d188"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.294659 17806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "840507dc2d394573958948e981180243"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "bd2c8663c7b0b014109b86cd3230c3ee"
server_key_iv: "c210e9cdd5be6e426ef8fd568403d188"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.296296 17806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:55.297356 17822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.297709 17806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:55.297830 17806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "840507dc2d394573958948e981180243"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "bd2c8663c7b0b014109b86cd3230c3ee"
server_key_iv: "c210e9cdd5be6e426ef8fd568403d188"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.297973 17806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:55.315727 17806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:55.318483 17806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:55.318651 17806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:55.319121 17806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:55.319494 17806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:55.319563 17806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.319607 17806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:55.319633 17806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.326013 17806 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45633
I20260807 15:04:55.326172 17942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45633 every 8 connection(s)
I20260807 15:04:55.326493 17806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:04:55.330600 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17806
I20260807 15:04:55.330735 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:04:55.331064 1535 external_mini_cluster.cc:1428] Setting key 9706ac49ed9a9a3e3ab1ace7181ae9c4
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:55.338312 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.327356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:55601 (local address 127.1.127.254:40173)
0807 15:04:55.327451 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.327453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.327633 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:04:55.328209 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.328210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.328224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.328284 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:55.329474 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.330318 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.331925 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.332147 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.332284 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.335092 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:55.335094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:55.335094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:55.335112 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:55.336723 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.337210 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.337212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.337212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.337280 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.337570 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.337571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.337572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.337706 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:55.337794 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.338100 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.338205 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:04:55.338550 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.327305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:55601)
0807 15:04:55.327501 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.327540 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.328078 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.328347 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.328350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.329116 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:55.329344 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.329350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.330485 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.330486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.331732 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.331735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.332299 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.332300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.332532 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.333195 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:55.333204 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:55.334973 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:55.336839 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.336849 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.336851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.337124 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.337363 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.337364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.337364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.337494 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.337783 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:55.337791 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:55.337955 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.338078 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.338265 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260807 15:04:55.339201 17943 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:04:55.339332 17943 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:55.339561 17943 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:04:55.340054 17604 ts_manager.cc:194] Registered new tserver with Master: 840507dc2d394573958948e981180243 (127.1.127.194:45633)
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:04:55.340541 17604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:55601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:04:55.362388 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:04:55.453014 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:04:55.453256 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:04:55.453284 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:04:55.454622 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:04:55.454728 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:04:55.454795 17948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:04:55.456533 17948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:04:55.456822 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:04:55.457082 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:04:55.462510 17956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.462560 17954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:04:55.462579 17953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:04:55.462970 17948 server_base.cc:1034] running on GCE node
I20260807 15:04:55.463212 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:04:55.463477 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:04:55.464675 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786115095464660 us; error 32 us; skew 500 ppm
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:04:55.467760 17948 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:04:55.468410 17948 webserver.cc:460] Webserver started at http://127.1.127.195:45493/ using document root <none> and password file <none>
I20260807 15:04:55.468676 17948 fs_manager.cc:359] Metadata directory not provided
I20260807 15:04:55.468757 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:04:55.468924 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:04:55.470136 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "b108bec5f0a74a3093282137b10d47d9"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "57371b19703e0a04fec28bfba1224574"
server_key_iv: "af4b9b2a53fc0b7a4b1da4d9f5e75a5e"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.470568 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "b108bec5f0a74a3093282137b10d47d9"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "57371b19703e0a04fec28bfba1224574"
server_key_iv: "af4b9b2a53fc0b7a4b1da4d9f5e75a5e"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.472179 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:04:55.473112 17963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.473358 17948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:04:55.473480 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "b108bec5f0a74a3093282137b10d47d9"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "57371b19703e0a04fec28bfba1224574"
server_key_iv: "af4b9b2a53fc0b7a4b1da4d9f5e75a5e"
server_key_version: "encryptionkey@0"
I20260807 15:04:55.473562 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:04:55.483309 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:04:55.485713 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:04:55.485921 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:04:55.486194 17948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:04:55.486534 17948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:04:55.486591 17948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.486634 17948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:04:55.486663 17948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:04:55.493225 17948 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:46439
I20260807 15:04:55.493271 18080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:46439 every 8 connection(s)
I20260807 15:04:55.493690 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:04:55.498908 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 17948
I20260807 15:04:55.499017 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:04:55.499224 1535 external_mini_cluster.cc:1428] Setting key 7d1d31335a14202ed4e8a1d18b086f5e
Aug 07 15:04:55 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115095, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:04:55.503928 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.494473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:51903 (local address 127.1.127.254:40173)
0807 15:04:55.494522 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.494524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.494735 (+ 211us) server_negotiation.cc:407] Connection header received
0807 15:04:55.495296 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.495297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.495308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.495376 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:55.496337 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.496909 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.497954 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.498144 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.498210 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.500821 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:55.500823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:55.500823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:55.500831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:55.502407 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.502896 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.502897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.502898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.502967 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.503235 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.503236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.503236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.503363 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:55.503437 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.503735 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.503824 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:04:55.504122 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.494410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:51903)
0807 15:04:55.494583 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.494627 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.495185 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.495472 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.495475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.496051 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:04:55.496246 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.496251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.496996 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.496997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.497857 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.497860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.498232 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.498233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.498377 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.499037 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:04:55.499048 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:04:55.500721 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:04:55.502497 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.502505 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.502508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.502816 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.503039 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:04:55.503040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:04:55.503041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:04:55.503162 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:04:55.503440 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:04:55.503448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:04:55.503593 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.503697 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.503866 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260807 15:04:55.504613 18081 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:04:55.504761 18081 heartbeater.cc:464] Registering TS with master...
I20260807 15:04:55.505000 18081 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:04:55.505491 17604 ts_manager.cc:194] Registered new tserver with Master: b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
I20260807 15:04:55.505946 17604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:51903
I20260807 15:04:55.511354 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:04:55.520107 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.512825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.1.127.254:40173)
0807 15:04:55.512871 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.512873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.513000 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:04:55.513081 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.513082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.513092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.513149 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:04:55.513724 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.514239 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.515102 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.515315 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.515399 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.516716 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:04:55.516717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:04:55.516717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:04:55.516726 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:04:55.518657 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.519120 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.519122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.519122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.519205 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:04:55.519422 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:04:55.519423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:04:55.519424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:04:55.519617 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:04:55.519738 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.519887 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.519997 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:04:55.522146 17604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:04:55.523900 17604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:04:55.532646 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.1.127.195:46439)
0807 15:04:55.527494 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.527500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.527755 (+ 255us) server_negotiation.cc:407] Connection header received
0807 15:04:55.527892 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.527895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.527914 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.527992 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:55.528360 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.529619 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.531345 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.531983 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532076 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532146 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.532510 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.532558 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
I20260807 15:04:55.532635 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:46439 (local address 127.0.0.1:47956)
0807 15:04:55.527679 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.527690 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.527786 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.528158 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.528159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.528161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:55.528254 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.528258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.529711 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.529712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.531246 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.531247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.532271 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.532365 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532371 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.532478 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.532531 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":24,"thread_start_us":140,"threads_started":1}
I20260807 15:04:55.532747 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35843 (local address 127.0.0.1:40252)
0807 15:04:55.527745 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.527757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.527820 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.528153 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.528156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.528159 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:55.528254 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.528257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.530724 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.530725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.531547 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.531550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.532494 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.532551 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532555 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.532587 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.532624 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260807 15:04:55.532541 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45633 (local address 127.0.0.1:37394)
0807 15:04:55.527363 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.527408 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.527482 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.528181 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.528182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.528185 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:55.528294 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.528296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.529680 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.529682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.531222 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.531225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.532196 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.532286 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532301 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.532373 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.532444 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":90,"thread_start_us":60,"threads_started":1}
I20260807 15:04:55.533070 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.1.127.194:45633)
0807 15:04:55.527717 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.527723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.527930 (+ 207us) server_negotiation.cc:407] Connection header received
0807 15:04:55.527987 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.527990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.528008 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.528087 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:55.528402 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.529570 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.531347 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.532026 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532102 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532157 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.532602 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.532649 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260807 15:04:55.533141 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.527682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.1.127.193:35843)
0807 15:04:55.527973 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.527977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.527996 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:04:55.528037 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.528039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.528054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.528132 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:55.529755 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.530574 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.531645 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.532400 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.532681 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.532931 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.532972 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.533037 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260807 15:04:55.533823 17872 tablet_service.cc:1467] Processing CreateTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d (DEFAULT_TABLE table=test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:55.533911 18014 tablet_service.cc:1467] Processing CreateTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d (DEFAULT_TABLE table=test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:55.534199 17872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f77d90e799431c8eb4a219bfe70a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:55.534343 18014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f77d90e799431c8eb4a219bfe70a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:55.537365 18099 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Bootstrap starting.
I20260807 15:04:55.537395 18100 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Bootstrap starting.
I20260807 15:04:55.538040 18100 tablet_bootstrap.cc:654] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:55.537959 17734 tablet_service.cc:1467] Processing CreateTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d (DEFAULT_TABLE table=test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:04:55.538192 18099 tablet_bootstrap.cc:654] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:55.538340 17734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f77d90e799431c8eb4a219bfe70a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:04:55.538882 18100 log.cc:824] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:55.538918 18099 log.cc:824] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:55.539749 18100 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: No bootstrap required, opened a new log
I20260807 15:04:55.539848 18100 ts_tablet_manager.cc:1397] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:04:55.539932 18099 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: No bootstrap required, opened a new log
I20260807 15:04:55.540037 18099 ts_tablet_manager.cc:1397] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:04:55.541587 18100 raft_consensus.cc:348] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.541787 18100 raft_consensus.cc:374] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:55.541862 18100 raft_consensus.cc:729] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840507dc2d394573958948e981180243, State: Initialized, Role: FOLLOWER
I20260807 15:04:55.542028 18100 consensus_queue.cc:260] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.542150 18099 raft_consensus.cc:348] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.542376 18100 ts_tablet_manager.cc:1428] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:04:55.542412 18099 raft_consensus.cc:374] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:55.542531 18099 raft_consensus.cc:729] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b108bec5f0a74a3093282137b10d47d9, State: Initialized, Role: FOLLOWER
I20260807 15:04:55.542582 17943 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
I20260807 15:04:55.542740 18103 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Bootstrap starting.
I20260807 15:04:55.542797 18099 consensus_queue.cc:260] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.543198 18099 ts_tablet_manager.cc:1428] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 15:04:55.543263 18081 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
I20260807 15:04:55.543985 18103 tablet_bootstrap.cc:654] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Neither blocks nor log segments found. Creating new log.
I20260807 15:04:55.544548 18103 log.cc:824] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:04:55.545454 18103 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: No bootstrap required, opened a new log
I20260807 15:04:55.545531 18103 ts_tablet_manager.cc:1397] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:04:55.547315 18103 raft_consensus.cc:348] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.547502 18103 raft_consensus.cc:374] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:04:55.547529 18103 raft_consensus.cc:729] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0373ae1509a486d80e6026dff5f83b6, State: Initialized, Role: FOLLOWER
I20260807 15:04:55.547679 18103 consensus_queue.cc:260] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.547999 18103 ts_tablet_manager.cc:1428] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 15:04:55.548094 17801 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
W20260807 15:04:55.577241 17944 tablet.cc:2369] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:04:55.645749 17802 tablet.cc:2369] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:04:55.701346 18107 raft_consensus.cc:482] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:04:55.701478 18107 raft_consensus.cc:504] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.701892 18107 leader_election.cc:288] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
I20260807 15:04:55.705431 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.702194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:48481 (local address 127.1.127.194:45633)
0807 15:04:55.702257 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.702259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.702359 (+ 100us) server_negotiation.cc:407] Connection header received
0807 15:04:55.702472 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.702474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.702518 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.702581 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:55.702832 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.703315 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.704622 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.705011 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.705114 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.705124 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.705317 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.705364 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:04:55.705422 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.702159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:46439 (local address 127.1.127.193:33157)
0807 15:04:55.702336 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.702348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.702447 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.702616 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.702617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.702619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:55.702723 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.702725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.703520 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.703521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.704352 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.704354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.705094 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.705095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.705174 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.705184 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.705228 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.705284 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260807 15:04:55.705408 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.702257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:33157 (local address 127.1.127.195:46439)
0807 15:04:55.702315 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.702317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.702428 (+ 111us) server_negotiation.cc:407] Connection header received
0807 15:04:55.702513 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.702515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.702524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.702607 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:04:55.702802 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.703349 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.704465 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.704958 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.705024 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.705032 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.705266 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.705320 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:04:55.705549 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.702103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45633 (local address 127.1.127.193:48481)
0807 15:04:55.702272 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.702285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:04:55.702368 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:04:55.702604 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:04:55.702605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:04:55.702607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:04:55.702738 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.702741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.703413 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.703414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.704520 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:04:55.704523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.705135 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.705136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:04:55.705202 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.705206 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:04:55.705293 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:04:55.705336 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":95,"thread_start_us":82,"threads_started":1}
I20260807 15:04:55.705890 18035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f77d90e799431c8eb4a219bfe70a8d" candidate_uuid: "e0373ae1509a486d80e6026dff5f83b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108bec5f0a74a3093282137b10d47d9" is_pre_election: true
I20260807 15:04:55.705930 17894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f77d90e799431c8eb4a219bfe70a8d" candidate_uuid: "e0373ae1509a486d80e6026dff5f83b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840507dc2d394573958948e981180243" is_pre_election: true
I20260807 15:04:55.706079 18035 raft_consensus.cc:2454] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0373ae1509a486d80e6026dff5f83b6 in term 0.
I20260807 15:04:55.706099 17894 raft_consensus.cc:2454] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0373ae1509a486d80e6026dff5f83b6 in term 0.
I20260807 15:04:55.706390 17685 leader_election.cc:302] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b108bec5f0a74a3093282137b10d47d9, e0373ae1509a486d80e6026dff5f83b6; no voters:
I20260807 15:04:55.706573 18107 raft_consensus.cc:2781] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:04:55.706656 18107 raft_consensus.cc:482] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:04:55.706712 18107 raft_consensus.cc:3037] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:55.707549 18107 raft_consensus.cc:504] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.707731 18107 leader_election.cc:288] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 1 election: Requested vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
I20260807 15:04:55.707907 17894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f77d90e799431c8eb4a219bfe70a8d" candidate_uuid: "e0373ae1509a486d80e6026dff5f83b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840507dc2d394573958948e981180243"
I20260807 15:04:55.707937 18035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f77d90e799431c8eb4a219bfe70a8d" candidate_uuid: "e0373ae1509a486d80e6026dff5f83b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108bec5f0a74a3093282137b10d47d9"
I20260807 15:04:55.708014 18035 raft_consensus.cc:3037] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:55.708003 17894 raft_consensus.cc:3037] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:04:55.709013 17894 raft_consensus.cc:2454] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0373ae1509a486d80e6026dff5f83b6 in term 1.
I20260807 15:04:55.709013 18035 raft_consensus.cc:2454] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0373ae1509a486d80e6026dff5f83b6 in term 1.
I20260807 15:04:55.709259 17685 leader_election.cc:302] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b108bec5f0a74a3093282137b10d47d9, e0373ae1509a486d80e6026dff5f83b6; no voters:
I20260807 15:04:55.709362 18107 raft_consensus.cc:2781] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:04:55.709533 18107 raft_consensus.cc:686] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 LEADER]: Becoming Leader. State: Replica: e0373ae1509a486d80e6026dff5f83b6, State: Running, Role: LEADER
I20260807 15:04:55.709658 18107 consensus_queue.cc:237] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:04:55.710753 17603 catalog_manager.cc:5495] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "e0373ae1509a486d80e6026dff5f83b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } health_report { overall_health: HEALTHY } } }
I20260807 15:04:55.719259 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.716229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.1.127.193:35843)
0807 15:04:55.716294 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:04:55.716296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:04:55.716305 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:04:55.716356 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:04:55.716356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:04:55.716375 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:04:55.716430 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:04:55.716644 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.717185 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.718156 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:04:55.718348 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:04:55.718416 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:04:55.718652 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:04:55.718980 (+ 328us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:04:55.719094 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:04:55.719137 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:04:55.719162 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:04:55.721443 17894 raft_consensus.cc:1261] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Refusing update from remote peer e0373ae1509a486d80e6026dff5f83b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:55.721443 18035 raft_consensus.cc:1261] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Refusing update from remote peer e0373ae1509a486d80e6026dff5f83b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:04:55.721807 18107 consensus_queue.cc:1035] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:55.722014 18107 consensus_queue.cc:1035] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:04:55.727591 18115 mvcc.cc:205] Tried to move back new op lower bound from 7315927432071946240 to 7315927432027611136. Current Snapshot: MvccSnapshot[applied={T|T < 7315927432071946240 or (T in {7315927432071946240})}]
I20260807 15:04:55.729715 17604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:55.729841 17604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:04:55.730892 17604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Sending DeleteTablet for 3 replicas of tablet e1f77d90e799431c8eb4a219bfe70a8d
I20260807 15:04:55.731307 17872 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:37394
I20260807 15:04:55.731364 17734 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:40252
I20260807 15:04:55.731379 18014 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:47956
I20260807 15:04:55.731599 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17669
I20260807 15:04:55.731624 18126 tablet_replica.cc:330] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: stopping tablet replica
I20260807 15:04:55.732859 18127 tablet_replica.cc:330] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: stopping tablet replica
I20260807 15:04:55.733003 18127 raft_consensus.cc:2229] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:55.733153 18127 raft_consensus.cc:2258] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:55.733675 18127 ts_tablet_manager.cc:1905] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:55.735659 18127 ts_tablet_manager.cc:1918] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:55.735747 18127 log.cc:1196] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/e1f77d90e799431c8eb4a219bfe70a8d
I20260807 15:04:55.735961 18124 tablet_replica.cc:330] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: stopping tablet replica
I20260807 15:04:55.736043 18127 ts_tablet_manager.cc:1939] T e1f77d90e799431c8eb4a219bfe70a8d P b108bec5f0a74a3093282137b10d47d9: Deleting consensus metadata
I20260807 15:04:55.736119 18124 raft_consensus.cc:2229] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:04:55.736245 18124 raft_consensus.cc:2258] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:04:55.736563 17588 catalog_manager.cc:4841] TS b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): tablet e1f77d90e799431c8eb4a219bfe70a8d (table test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]) successfully deleted
I20260807 15:04:55.736742 18124 ts_tablet_manager.cc:1905] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:04:55.738880 18124 ts_tablet_manager.cc:1918] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:04:55.738986 18124 log.cc:1196] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/e1f77d90e799431c8eb4a219bfe70a8d
I20260807 15:04:55.739312 18124 ts_tablet_manager.cc:1939] T e1f77d90e799431c8eb4a219bfe70a8d P 840507dc2d394573958948e981180243: Deleting consensus metadata
I20260807 15:04:55.740166 17591 catalog_manager.cc:4841] TS 840507dc2d394573958948e981180243 (127.1.127.194:45633): tablet e1f77d90e799431c8eb4a219bfe70a8d (table test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]) successfully deleted
W20260807 15:04:55.741696 17588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:35843 (error 108)
I20260807 15:04:55.742051 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17806
I20260807 15:04:55.744050 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:04:55.743879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35843 (local address 127.0.0.1:40276)
0807 15:04:55.743955 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 15:04:55.744012 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:04:55.744251 17588 catalog_manager.cc:4584] TS e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1f77d90e799431c8eb4a219bfe70a8d: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35843: connect: Connection refused (error 111)
I20260807 15:04:55.749080 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17948
I20260807 15:04:55.756177 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 17572
I20260807 15:05:15.764357 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--webserver_interface=127.1.127.254
--webserver_port=45631
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:15.852803 18129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:15.853066 18129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:15.853093 18129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:15.854384 18129 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:05:15.854439 18129 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:05:15.854452 18129 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:05:15.854465 18129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:15.854477 18129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:15.854491 18129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:15.854538 18129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:15.854557 18129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:15.856293 18129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:40173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:15.856583 18129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:15.856850 18129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:05:15.867331 18129 server_base.cc:1034] running on GCE node
W20260807 15:05:15.867311 18136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:15.867298 18140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:15.867298 18137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:15.867872 18129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:15.868201 18129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:15.869351 18129 hybrid_clock.cc:648] HybridClock initialized: now 1786115115869337 us; error 31 us; skew 500 ppm
Aug 07 15:05:15 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115115, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:15.872177 18129 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:05:15.872732 18129 webserver.cc:460] Webserver started at http://127.1.127.254:45631/ using document root <none> and password file <none>
I20260807 15:05:15.873000 18129 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:15.873065 18129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:15.874608 18129 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:05:15.875479 18147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:15.875725 18129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:15.875834 18129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee"
format_stamp: "Formatted at 2026-08-07 15:04:54 on dist-test-slave-8tdl"
server_key: "499fc365cfc50a9b11b054106860a411"
server_key_iv: "2f7f7781e3be57d755cf6e2360eaa888"
server_key_version: "encryptionkey@0"
I20260807 15:05:15.876281 18129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:15.907605 18129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:15.910203 18129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:15.910409 18129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:15.915421 18129 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:40173
I20260807 15:05:15.915453 18201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:40173 every 8 connection(s)
I20260807 15:05:15.915908 18129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:15.917734 18202 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:05:15.918468 18202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap starting.
I20260807 15:05:15.921527 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18129
I20260807 15:05:15.921893 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:35843
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=42097
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:05:15.929235 18202 log.cc:824] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:15.931612 18202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:05:15.932006 18202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap complete.
I20260807 15:05:15.934789 18202 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:15.935132 18202 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Initialized, Role: FOLLOWER
I20260807 15:05:15.935328 18202 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:15.935417 18202 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:15.935484 18202 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:15.935544 18202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:05:15.936739 18202 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:15.936885 18202 leader_election.cc:302] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab3d11a7d8184879bfaacc2a5e7c9cee; no voters:
I20260807 15:05:15.937114 18202 leader_election.cc:288] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:05:15.937166 18206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:05:15.937361 18206 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 LEADER]: Becoming Leader. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Running, Role: LEADER
I20260807 15:05:15.937487 18202 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:15.937525 18206 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:15.938021 18208 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab3d11a7d8184879bfaacc2a5e7c9cee. Latest consensus state: current_term: 2 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:05:15.938503 18208 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:15.938719 18207 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:05:15.938792 18216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:15.938872 18207 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:15.939857 18216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:15.940054 18216 catalog_manager.cc:1259] Loaded cluster ID: a8cdb8c5dd2e4a53ba78c94aeebf8035
I20260807 15:05:15.940287 18216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:15.941634 18216 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:15.941850 18216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Loaded TSK: 0
I20260807 15:05:15.955732 18216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Generated new TSK 1
I20260807 15:05:15.955883 18216 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:05:16.016826 18204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:16.017086 18204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:16.017114 18204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:16.018469 18204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:16.018527 18204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:16.018558 18204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:16.020223 18204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:35843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=42097
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:16.020488 18204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:16.020753 18204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:16.026417 18230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.026396 18229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.026397 18232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:16.026947 18204 server_base.cc:1034] running on GCE node
I20260807 15:05:16.027190 18204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:16.027580 18204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:16.028784 18204 hybrid_clock.cc:648] HybridClock initialized: now 1786115116028768 us; error 34 us; skew 500 ppm
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:16.031931 18204 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:05:16.032518 18204 webserver.cc:460] Webserver started at http://127.1.127.193:42097/ using document root <none> and password file <none>
I20260807 15:05:16.032783 18204 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:16.032855 18204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:16.034508 18204 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:05:16.035478 18239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.035708 18204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:16.035822 18204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "e0373ae1509a486d80e6026dff5f83b6"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "4c6736413e6f3b762450bbe3fdf14803"
server_key_iv: "7a9a06b4df13afb31d120a6ac28e0491"
server_key_version: "encryptionkey@0"
I20260807 15:05:16.036273 18204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:16.054041 18204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:16.056769 18204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:16.057054 18204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:16.057346 18204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:16.057847 18246 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:05:16.059916 18204 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:05:16.059983 18204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:05:16.060012 18204 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:05:16.060767 18204 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:05:16.060825 18204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:05:16.060909 18246 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Bootstrap starting.
I20260807 15:05:16.068647 18204 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35843
I20260807 15:05:16.068817 18358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35843 every 8 connection(s)
I20260807 15:05:16.069128 18204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:05:16.073117 18246 log.cc:824] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:16.076083 18246 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:05:16.076467 18246 tablet_bootstrap.cc:492] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Bootstrap complete.
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:16.077970 18246 ts_tablet_manager.cc:1397] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20260807 15:05:16.078369 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18204
I20260807 15:05:16.078668 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:45633
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=42641
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:05:16.079509 18246 raft_consensus.cc:348] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.079866 18246 raft_consensus.cc:729] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0373ae1509a486d80e6026dff5f83b6, State: Initialized, Role: FOLLOWER
I20260807 15:05:16.080044 18246 consensus_queue.cc:260] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.080433 18246 ts_tablet_manager.cc:1428] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20260807 15:05:16.082187 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.069913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:59953)
0807 15:05:16.070127 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.070183 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.070729 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.071242 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.071247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.071549 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:16.071739 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.071744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.072966 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.072967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.074079 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.074082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.074769 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.074770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.074916 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.075570 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:16.075579 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:16.077113 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:16.080255 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.080263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.080265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.080699 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.080988 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.080989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.080990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.081109 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.081512 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:16.081520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:16.081663 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.081776 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.081983 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260807 15:05:16.082186 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.070005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:59953 (local address 127.1.127.254:40173)
0807 15:05:16.070842 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.070852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.070895 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:05:16.071021 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.071026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.071068 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.071210 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:16.071863 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.072796 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.074178 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.074619 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.074776 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.077252 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:16.077256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:16.077257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:16.077270 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:16.079834 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.080789 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.080792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.080793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.080908 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.081179 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.081180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.081180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.081430 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:16.081532 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.081850 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.081988 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":113,"threads_started":1}
I20260807 15:05:16.083360 18359 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:05:16.083472 18359 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:16.083659 18359 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:05:16.084425 18165 ts_manager.cc:194] Registered new tserver with Master: e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843)
I20260807 15:05:16.085574 18165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:59953
I20260807 15:05:16.086450 18359 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
I20260807 15:05:16.094310 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.085029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35843 (local address 127.0.0.1:38074)
0807 15:05:16.090875 (+ 5846us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.090928 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.091075 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.091449 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.091453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.091455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:16.091547 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.091550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.092428 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.092430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.093282 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.093284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.093992 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.093993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.094086 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.094102 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.094177 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.094239 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5803,"thread_start_us":104,"threads_started":1}
I20260807 15:05:16.094745 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.085297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.1.127.193:35843)
0807 15:05:16.091282 (+ 5985us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.091286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.091296 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:16.091331 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.091333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.091348 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.091464 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:16.091624 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.092333 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.093386 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.093895 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.093990 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.094058 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.094578 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.094628 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5934,"thread_start_us":76,"threads_started":1}
I20260807 15:05:16.095160 18290 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f77d90e799431c8eb4a219bfe70a8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:04:55 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:38074
I20260807 15:05:16.095741 18368 tablet_replica.cc:330] stopping tablet replica
I20260807 15:05:16.095868 18368 raft_consensus.cc:2229] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:16.095969 18368 raft_consensus.cc:2258] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:16.096311 18368 ts_tablet_manager.cc:1905] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:16.098788 18368 ts_tablet_manager.cc:1918] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:16.098881 18368 log.cc:1196] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal/wals/e1f77d90e799431c8eb4a219bfe70a8d
I20260807 15:05:16.099176 18368 ts_tablet_manager.cc:1939] T e1f77d90e799431c8eb4a219bfe70a8d P e0373ae1509a486d80e6026dff5f83b6: Deleting consensus metadata
I20260807 15:05:16.099759 18148 catalog_manager.cc:4841] TS e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843): tablet e1f77d90e799431c8eb4a219bfe70a8d (table test-table [id=3ac3b5f970e74b3696e8a8ca5c5dc883]) successfully deleted
W20260807 15:05:16.170845 18365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:16.171064 18365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:16.171092 18365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:16.172307 18365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:16.172351 18365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:16.172379 18365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:16.173980 18365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:45633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=42641
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:16.174245 18365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:16.174490 18365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:16.179709 18377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.179721 18373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.179750 18375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:16.179915 18365 server_base.cc:1034] running on GCE node
I20260807 15:05:16.180130 18365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:16.180415 18365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:16.181588 18365 hybrid_clock.cc:648] HybridClock initialized: now 1786115116181577 us; error 38 us; skew 500 ppm
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:16.184322 18365 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:05:16.184805 18365 webserver.cc:460] Webserver started at http://127.1.127.194:42641/ using document root <none> and password file <none>
I20260807 15:05:16.185056 18365 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:16.185145 18365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:16.187805 18365 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:05:16.188570 18384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.188803 18365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:16.188905 18365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "840507dc2d394573958948e981180243"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "bd2c8663c7b0b014109b86cd3230c3ee"
server_key_iv: "c210e9cdd5be6e426ef8fd568403d188"
server_key_version: "encryptionkey@0"
I20260807 15:05:16.189281 18365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:16.201861 18365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:16.204655 18365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:16.204800 18365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:16.205018 18365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:16.205391 18365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:16.205454 18365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.205490 18365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:16.205533 18365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.211931 18365 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45633
I20260807 15:05:16.211973 18503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45633 every 8 connection(s)
I20260807 15:05:16.212366 18365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:05:16.216409 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18365
I20260807 15:05:16.216745 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:46439
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=45493
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:16.224252 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.213157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:34713 (local address 127.1.127.254:40173)
0807 15:05:16.213242 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.213244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.213480 (+ 236us) server_negotiation.cc:407] Connection header received
0807 15:05:16.214030 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.214032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.214047 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.214106 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:16.215256 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.215823 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.217405 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.217648 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.217714 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.220725 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:16.220728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:16.220728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:16.220736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:16.222267 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.222922 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.222923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.222923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.222991 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.223382 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.223383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.223383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.223515 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:16.223579 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.224075 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.224177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:05:16.224654 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.213123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:34713)
0807 15:05:16.213318 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.213359 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.213850 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.214172 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.214176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.214896 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:16.215113 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.215118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.215982 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.215983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.217308 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.217311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.217743 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.217744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.217914 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.218634 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:16.218646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:16.220599 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:16.222377 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.222388 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.222391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.222832 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.223076 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.223077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.223077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.223264 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.223597 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:16.223604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:16.223817 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.223957 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.224212 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260807 15:05:16.225347 18505 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:05:16.225484 18505 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:16.225732 18505 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:05:16.226179 18165 ts_manager.cc:194] Registered new tserver with Master: 840507dc2d394573958948e981180243 (127.1.127.194:45633)
I20260807 15:05:16.226590 18165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:34713
W20260807 15:05:16.307085 18508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:16.307340 18508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:16.307376 18508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:16.308656 18508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:16.308703 18508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:16.308732 18508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:16.310305 18508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:46439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=45493
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:16.310547 18508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:16.310772 18508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:16.316516 18513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.316574 18514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:16.316517 18516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:16.316916 18508 server_base.cc:1034] running on GCE node
I20260807 15:05:16.317209 18508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:16.317513 18508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:16.318734 18508 hybrid_clock.cc:648] HybridClock initialized: now 1786115116318720 us; error 42 us; skew 500 ppm
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:16.321977 18508 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:05:16.322568 18508 webserver.cc:460] Webserver started at http://127.1.127.195:45493/ using document root <none> and password file <none>
I20260807 15:05:16.322861 18508 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:16.322947 18508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:16.324560 18508 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:05:16.325497 18524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.325785 18508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:16.325932 18508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "b108bec5f0a74a3093282137b10d47d9"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "57371b19703e0a04fec28bfba1224574"
server_key_iv: "af4b9b2a53fc0b7a4b1da4d9f5e75a5e"
server_key_version: "encryptionkey@0"
I20260807 15:05:16.326398 18508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:16.339465 18508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:16.342236 18508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:16.342442 18508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:16.342768 18508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:16.343146 18508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:16.343184 18508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.343232 18508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:16.343254 18508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.350364 18508 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:46439
I20260807 15:05:16.350471 18641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:46439 every 8 connection(s)
I20260807 15:05:16.350900 18508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:16.354022 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18508
Aug 07 15:05:16 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:16.363440 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.354539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.1.127.254:40173)
0807 15:05:16.354719 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.354721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.354816 (+ 95us) server_negotiation.cc:407] Connection header received
0807 15:05:16.354874 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.354875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.354888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.354974 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:16.355564 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.356310 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.357369 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.357670 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.357751 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.359612 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:16.359613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:16.359614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:16.359621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:16.361666 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.362274 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.362290 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.362290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.362378 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.362634 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.362634 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.362635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.362835 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:16.363018 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.363202 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.363310 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260807 15:05:16.363610 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.351850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:59059 (local address 127.1.127.254:40173)
0807 15:05:16.351933 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.351935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.352121 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:05:16.352659 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.352660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.352672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.352771 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:16.353929 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.354614 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.355992 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.356207 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.356277 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.359366 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:16.359368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:16.359368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:16.359376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:16.361698 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.362315 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.362316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.362316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.362392 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.362762 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.362763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.362764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.362866 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:16.362930 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.363318 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.363385 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":50}
I20260807 15:05:16.363844 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.351802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:59059)
0807 15:05:16.351981 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.352026 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.352530 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.352827 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.352830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.353582 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:16.353803 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.353809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.354748 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.354749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.355869 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.355874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.356312 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.356313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.356488 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.357123 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:16.357133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:16.359226 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:16.361816 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.361834 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.361837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.362242 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.362488 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:16.362488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:16.362489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:16.362659 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:16.362944 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:16.362953 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:16.363157 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.363290 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.363571 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260807 15:05:16.364383 18642 heartbeater.cc:347] Connected to a master server at 127.1.127.254:40173
I20260807 15:05:16.364526 18642 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:16.364733 18642 heartbeater.cc:511] Master 127.1.127.254:40173 requested a full tablet report, sending...
I20260807 15:05:16.365125 18165 ts_manager.cc:194] Registered new tserver with Master: b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
I20260807 15:05:16.365471 18165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:59059
I20260807 15:05:16.375912 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:05:16.379344 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.376413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.1.127.254:40173)
0807 15:05:16.376580 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.376582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.376639 (+ 57us) server_negotiation.cc:407] Connection header received
0807 15:05:16.376710 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.376711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.376749 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.376817 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:16.377098 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.377603 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.378451 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.378810 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.378897 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.379057 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:16.379123 (+ 66us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 15:05:16.379287 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50662: token expired
Metrics: {"server-negotiator.queue_time_us":140}
W20260807 15:05:16.379426 18646 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50662: token expired
W20260807 15:05:16.380510 18085 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.127.254:40173)
I20260807 15:05:16.387368 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.380851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.1.127.254:40173)
0807 15:05:16.380917 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.380918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.380957 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:05:16.381021 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.381022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.381031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.381089 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:16.381630 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.382104 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.382924 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.383123 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.383178 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.384540 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:16.384541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:16.384542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:16.384549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:16.386054 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.386421 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.386422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.386422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.386479 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:16.386668 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:16.386669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:16.386669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:16.386861 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:16.386943 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.387155 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.387227 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:05:16.388849 18165 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:05:16.390616 18165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:05:16.393700 18290 tablet_service.cc:1467] Processing CreateTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 (DEFAULT_TABLE table=test-table [id=1be14a5ef1e94b7986de993c273dc664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:16.393962 18290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1840021b7bf4e7dbb18b5a103e46bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:16.395546 18246 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Bootstrap starting.
I20260807 15:05:16.395925 18246 tablet_bootstrap.cc:654] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:16.396497 18246 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: No bootstrap required, opened a new log
I20260807 15:05:16.396551 18246 ts_tablet_manager.cc:1397] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:05:16.396688 18246 raft_consensus.cc:348] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.396759 18246 raft_consensus.cc:374] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:16.396828 18246 raft_consensus.cc:729] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0373ae1509a486d80e6026dff5f83b6, State: Initialized, Role: FOLLOWER
I20260807 15:05:16.396883 18246 consensus_queue.cc:260] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.396976 18246 ts_tablet_manager.cc:1428] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:16.397382 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.393031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:46439 (local address 127.0.0.1:41178)
0807 15:05:16.393253 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.393263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.393304 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.393703 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.393705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.393707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:16.393756 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.393757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.395082 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.395083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.396129 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.396132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.397136 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.397136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.397195 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.397198 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.397244 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.397278 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":27,"thread_start_us":101,"threads_started":1}
I20260807 15:05:16.397388 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.393031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45633 (local address 127.0.0.1:34322)
0807 15:05:16.393096 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.393111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.393165 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.393570 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.393571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.393574 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:16.393626 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.393627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.394658 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.394660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.395761 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.395764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.397093 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.397095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.397162 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.397166 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.397198 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.397256 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:05:16.397660 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.393242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.1.127.195:46439)
0807 15:05:16.393441 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.393444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.393453 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:05:16.393498 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.393500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.393516 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.393592 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:16.393840 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.394987 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.396230 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.396961 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.397034 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.397104 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.397515 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.397576 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260807 15:05:16.397715 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.393166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.1.127.194:45633)
0807 15:05:16.393349 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.393355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.393369 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:05:16.393417 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.393420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.393438 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.393568 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:16.393686 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.394564 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.395922 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.396957 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.397337 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.397405 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.397436 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.397620 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260807 15:05:16.398492 18573 tablet_service.cc:1467] Processing CreateTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 (DEFAULT_TABLE table=test-table [id=1be14a5ef1e94b7986de993c273dc664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:16.398494 18433 tablet_service.cc:1467] Processing CreateTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 (DEFAULT_TABLE table=test-table [id=1be14a5ef1e94b7986de993c273dc664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:16.398871 18573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1840021b7bf4e7dbb18b5a103e46bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:16.398880 18433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1840021b7bf4e7dbb18b5a103e46bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:16.401852 18652 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Bootstrap starting.
I20260807 15:05:16.401861 18653 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Bootstrap starting.
I20260807 15:05:16.402544 18653 tablet_bootstrap.cc:654] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:16.402575 18652 tablet_bootstrap.cc:654] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:16.403129 18652 log.cc:824] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:16.403132 18653 log.cc:824] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:16.403975 18653 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: No bootstrap required, opened a new log
I20260807 15:05:16.403982 18652 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: No bootstrap required, opened a new log
I20260807 15:05:16.404064 18653 ts_tablet_manager.cc:1397] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:05:16.404078 18652 ts_tablet_manager.cc:1397] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:05:16.405484 18652 raft_consensus.cc:348] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.405488 18653 raft_consensus.cc:348] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.405640 18652 raft_consensus.cc:374] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:16.405666 18652 raft_consensus.cc:729] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b108bec5f0a74a3093282137b10d47d9, State: Initialized, Role: FOLLOWER
I20260807 15:05:16.405664 18653 raft_consensus.cc:374] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:16.405707 18653 raft_consensus.cc:729] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840507dc2d394573958948e981180243, State: Initialized, Role: FOLLOWER
I20260807 15:05:16.405798 18652 consensus_queue.cc:260] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.405838 18653 consensus_queue.cc:260] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.406126 18642 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
I20260807 15:05:16.406152 18653 ts_tablet_manager.cc:1428] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:05:16.406164 18505 heartbeater.cc:503] Master 127.1.127.254:40173 was elected leader, sending a full tablet report...
I20260807 15:05:16.406123 18652 ts_tablet_manager.cc:1428] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260807 15:05:16.463016 18506 tablet.cc:2369] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:16.563079 18656 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:16.563226 18656 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.563591 18656 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843)
I20260807 15:05:16.566444 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.563882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:60735 (local address 127.1.127.194:45633)
0807 15:05:16.563950 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.563952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.563963 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:05:16.563996 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.563996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.564010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.564085 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:16.564249 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.564766 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.565698 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.566083 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.566172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.566181 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.566338 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.566376 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:05:16.566444 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.563799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45633 (local address 127.1.127.195:60735)
0807 15:05:16.563862 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.563875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.563922 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.564092 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.564094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.564095 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:16.564177 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.564179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.564860 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.564861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.565602 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.565604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.566171 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.566172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.566256 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.566260 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.566319 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.566363 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:05:16.566716 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.563856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35843 (local address 127.1.127.195:44457)
0807 15:05:16.564110 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.564119 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:16.564200 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:16.564348 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:16.564349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:16.564350 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:16.564432 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.564433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.565090 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.565091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.565850 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:16.565852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.566396 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.566398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:16.566476 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.566479 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:16.566543 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:05:16.566588 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":191,"threads_started":1}
I20260807 15:05:16.566833 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.563894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:44457 (local address 127.1.127.193:35843)
0807 15:05:16.563953 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.563956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.564189 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:05:16.564269 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.564270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.564279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.564343 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:16.564510 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.564996 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.565956 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.566315 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.566398 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.566406 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.566700 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.566753 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:05:16.566847 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1840021b7bf4e7dbb18b5a103e46bb3" candidate_uuid: "b108bec5f0a74a3093282137b10d47d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840507dc2d394573958948e981180243" is_pre_election: true
I20260807 15:05:16.567018 18454 raft_consensus.cc:2454] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b108bec5f0a74a3093282137b10d47d9 in term 0.
I20260807 15:05:16.567101 18312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1840021b7bf4e7dbb18b5a103e46bb3" candidate_uuid: "b108bec5f0a74a3093282137b10d47d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0373ae1509a486d80e6026dff5f83b6" is_pre_election: true
I20260807 15:05:16.567269 18528 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9; no voters:
I20260807 15:05:16.567298 18312 raft_consensus.cc:2454] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b108bec5f0a74a3093282137b10d47d9 in term 0.
I20260807 15:05:16.567412 18656 raft_consensus.cc:2781] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:05:16.567497 18656 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:05:16.567523 18656 raft_consensus.cc:3037] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:16.568252 18656 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.568400 18656 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [CANDIDATE]: Term 1 election: Requested vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843)
I20260807 15:05:16.568596 18312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1840021b7bf4e7dbb18b5a103e46bb3" candidate_uuid: "b108bec5f0a74a3093282137b10d47d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0373ae1509a486d80e6026dff5f83b6"
I20260807 15:05:16.568614 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1840021b7bf4e7dbb18b5a103e46bb3" candidate_uuid: "b108bec5f0a74a3093282137b10d47d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840507dc2d394573958948e981180243"
I20260807 15:05:16.568691 18454 raft_consensus.cc:3037] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:16.568691 18312 raft_consensus.cc:3037] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:16.569763 18454 raft_consensus.cc:2454] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b108bec5f0a74a3093282137b10d47d9 in term 1.
I20260807 15:05:16.569983 18312 raft_consensus.cc:2454] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b108bec5f0a74a3093282137b10d47d9 in term 1.
I20260807 15:05:16.570084 18528 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9; no voters:
I20260807 15:05:16.570228 18656 raft_consensus.cc:2781] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Leader election won for term 1
W20260807 15:05:16.570267 18360 tablet.cc:2369] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:16.570377 18656 raft_consensus.cc:686] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 1 LEADER]: Becoming Leader. State: Replica: b108bec5f0a74a3093282137b10d47d9, State: Running, Role: LEADER
I20260807 15:05:16.570447 18656 consensus_queue.cc:237] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:16.571560 18165 catalog_manager.cc:5495] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b108bec5f0a74a3093282137b10d47d9 (127.1.127.195). New cstate: current_term: 1 leader_uuid: "b108bec5f0a74a3093282137b10d47d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } health_report { overall_health: UNKNOWN } } }
I20260807 15:05:16.585302 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.582590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.1.127.195:46439)
0807 15:05:16.582654 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:16.582656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:16.582666 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:16.582749 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:16.582749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:16.582769 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:16.582832 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:16.583073 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.583553 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.584348 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:16.584560 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:16.584629 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:16.584827 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:16.584959 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:05:16.585028 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:16.585190 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:05:16.585213 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:05:16.587417 18312 raft_consensus.cc:1261] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Refusing update from remote peer b108bec5f0a74a3093282137b10d47d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:16.587417 18454 raft_consensus.cc:1261] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Refusing update from remote peer b108bec5f0a74a3093282137b10d47d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:16.587816 18656 consensus_queue.cc:1035] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:16.588019 18659 consensus_queue.cc:1035] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:16.590431 18662 mvcc.cc:205] Tried to move back new op lower bound from 7315927517539057664 to 7315927517473320960. Current Snapshot: MvccSnapshot[applied={T|T < 7315927517539057664}]
I20260807 15:05:16.590565 18663 mvcc.cc:205] Tried to move back new op lower bound from 7315927517539057664 to 7315927517473320960. Current Snapshot: MvccSnapshot[applied={T|T < 7315927517539057664}]
I20260807 15:05:16.591436 18665 mvcc.cc:205] Tried to move back new op lower bound from 7315927517539057664 to 7315927517473320960. Current Snapshot: MvccSnapshot[applied={T|T < 7315927517539057664}]
I20260807 15:05:16.593432 18165 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:16.593532 18165 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:16.594445 18165 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Sending DeleteTablet for 3 replicas of tablet f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:16.594830 18433 tablet_service.cc:1514] Processing DeleteTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:16 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:34322
I20260807 15:05:16.594828 18573 tablet_service.cc:1514] Processing DeleteTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:16 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:41178
I20260807 15:05:16.594897 18290 tablet_service.cc:1514] Processing DeleteTablet for tablet f1840021b7bf4e7dbb18b5a103e46bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:16 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:38074
I20260807 15:05:16.594991 18368 tablet_replica.cc:330] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: stopping tablet replica
I20260807 15:05:16.595048 18368 raft_consensus.cc:2229] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:16.595103 18368 raft_consensus.cc:2258] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:16.595105 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18204
I20260807 15:05:16.597065 18673 tablet_replica.cc:330] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: stopping tablet replica
I20260807 15:05:16.597191 18673 raft_consensus.cc:2229] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:16.597314 18673 raft_consensus.cc:2258] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:16.597501 18674 tablet_replica.cc:330] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: stopping tablet replica
I20260807 15:05:16.597615 18674 raft_consensus.cc:2229] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:05:16.597726 18674 raft_consensus.cc:2258] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:16.598193 18674 ts_tablet_manager.cc:1905] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:16.600219 18674 ts_tablet_manager.cc:1918] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:16.600317 18674 log.cc:1196] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:16.600414 18673 ts_tablet_manager.cc:1905] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:16.600692 18674 ts_tablet_manager.cc:1939] T f1840021b7bf4e7dbb18b5a103e46bb3 P b108bec5f0a74a3093282137b10d47d9: Deleting consensus metadata
I20260807 15:05:16.601209 18148 catalog_manager.cc:4841] TS b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): tablet f1840021b7bf4e7dbb18b5a103e46bb3 (table test-table [id=1be14a5ef1e94b7986de993c273dc664]) successfully deleted
I20260807 15:05:16.602123 18673 ts_tablet_manager.cc:1918] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:16.602202 18673 log.cc:1196] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:16.602505 18673 ts_tablet_manager.cc:1939] T f1840021b7bf4e7dbb18b5a103e46bb3 P 840507dc2d394573958948e981180243: Deleting consensus metadata
I20260807 15:05:16.603111 18151 catalog_manager.cc:4841] TS 840507dc2d394573958948e981180243 (127.1.127.194:45633): tablet f1840021b7bf4e7dbb18b5a103e46bb3 (table test-table [id=1be14a5ef1e94b7986de993c273dc664]) successfully deleted
W20260807 15:05:16.604110 18148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:35843 (error 108)
I20260807 15:05:16.604573 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18365
I20260807 15:05:16.604691 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:16.604509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:35843 (local address 127.0.0.1:38084)
0807 15:05:16.604574 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:16.604647 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 15:05:16.606383 18148 catalog_manager.cc:4584] TS e0373ae1509a486d80e6026dff5f83b6 (127.1.127.193:35843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1840021b7bf4e7dbb18b5a103e46bb3: Network error: Client connection negotiation failed: client connection to 127.1.127.193:35843: connect: Connection refused (error 111)
I20260807 15:05:16.612526 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18508
I20260807 15:05:16.619483 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18129
I20260807 15:05:36.627213 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--webserver_interface=127.1.127.254
--webserver_port=45631
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:36.716339 18676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:36.716622 18676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:36.716682 18676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:36.718067 18676 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:05:36.718158 18676 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:05:36.718179 18676 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:05:36.718200 18676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:36.718232 18676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:36.718264 18676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:36.718294 18676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:36.718318 18676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:36.720235 18676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:40173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:36.720608 18676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:36.720872 18676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:36.727820 18684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:36.727799 18682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:36.727852 18676 server_base.cc:1034] running on GCE node
W20260807 15:05:36.727803 18681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:36.728349 18676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:36.728675 18676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:36.729919 18676 hybrid_clock.cc:648] HybridClock initialized: now 1786115136729806 us; error 48 us; skew 500 ppm
Aug 07 15:05:36 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115136, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:36.732887 18676 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:05:36.733496 18676 webserver.cc:460] Webserver started at http://127.1.127.254:45631/ using document root <none> and password file <none>
I20260807 15:05:36.733701 18676 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:36.733744 18676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:36.735386 18676 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:05:36.736368 18691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:36.736902 18676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:36.737157 18676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee"
format_stamp: "Formatted at 2026-08-07 15:04:54 on dist-test-slave-8tdl"
server_key: "499fc365cfc50a9b11b054106860a411"
server_key_iv: "2f7f7781e3be57d755cf6e2360eaa888"
server_key_version: "encryptionkey@0"
I20260807 15:05:36.737566 18676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:36.750304 18676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:36.752796 18676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:36.752924 18676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:36.757443 18676 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:40173
I20260807 15:05:36.757524 18745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:40173 every 8 connection(s)
I20260807 15:05:36.757930 18676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:36.759665 18746 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:05:36.760491 18746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap starting.
I20260807 15:05:36.764004 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18676
I20260807 15:05:36.764330 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:35843
--local_ip_for_outbound_sockets=127.1.127.193
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=42097
--webserver_interface=127.1.127.193
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:05:36.770467 18746 log.cc:824] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:36.772975 18746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:05:36.773371 18746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Bootstrap complete.
I20260807 15:05:36.775993 18746 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:36.776271 18746 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Initialized, Role: FOLLOWER
I20260807 15:05:36.776474 18746 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:36.776561 18746 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:36.776618 18746 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:36.776667 18746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:05:36.777777 18746 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:36.777917 18746 leader_election.cc:302] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab3d11a7d8184879bfaacc2a5e7c9cee; no voters:
I20260807 15:05:36.778146 18746 leader_election.cc:288] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:05:36.778196 18751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:05:36.778378 18751 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [term 3 LEADER]: Becoming Leader. State: Replica: ab3d11a7d8184879bfaacc2a5e7c9cee, State: Running, Role: LEADER
I20260807 15:05:36.778462 18746 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:36.778537 18751 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } }
I20260807 15:05:36.778949 18751 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab3d11a7d8184879bfaacc2a5e7c9cee. Latest consensus state: current_term: 3 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:05:36.779057 18752 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab3d11a7d8184879bfaacc2a5e7c9cee" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40173 } } }
I20260807 15:05:36.779230 18752 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:36.779117 18751 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:36.779528 18760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:36.780519 18760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:36.780705 18760 catalog_manager.cc:1259] Loaded cluster ID: a8cdb8c5dd2e4a53ba78c94aeebf8035
I20260807 15:05:36.780923 18760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:36.782135 18760 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:36.782316 18760 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Loaded TSK: 0
I20260807 15:05:36.782366 18760 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab3d11a7d8184879bfaacc2a5e7c9cee: Loaded TSK: 1
F20260807 15:05:36.782598 18760 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c1a3b29ad google::LogMessage::Fail() at ??:0
@ 0x7f2c1a3b588a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c1a3b24de google::LogMessage::Flush() at ??:0
@ 0x7f2c1a3b3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c1cb1f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2c1cb1cd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2c2230c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2c2230c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2c2230c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2c222d2c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2c222dcad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c1c2800d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c1c275954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c1d0d96db start_thread at ??:0
@ 0x7f2c18eb061f clone at ??:0
W20260807 15:05:36.874428 18748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:36.874882 18748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:36.874928 18748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:36.876283 18748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:36.876331 18748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:36.876360 18748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:36.877997 18748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:35843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=42097
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:36.878274 18748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:36.878562 18748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:05:36.884589 18748 server_base.cc:1034] running on GCE node
W20260807 15:05:36.884763 18789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:36.884720 18790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:36.884590 18792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:36.885169 18748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:36.885495 18748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:36.886745 18748 hybrid_clock.cc:648] HybridClock initialized: now 1786115136886708 us; error 43 us; skew 500 ppm
Aug 07 15:05:36 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115136, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:36.889576 18748 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:05:36.890182 18748 webserver.cc:460] Webserver started at http://127.1.127.193:42097/ using document root <none> and password file <none>
I20260807 15:05:36.890475 18748 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:36.890524 18748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:36.892103 18748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:36.892997 18799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:36.893298 18748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:36.893430 18748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "e0373ae1509a486d80e6026dff5f83b6"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "4c6736413e6f3b762450bbe3fdf14803"
server_key_iv: "7a9a06b4df13afb31d120a6ac28e0491"
server_key_version: "encryptionkey@0"
I20260807 15:05:36.893893 18748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:36.914322 18748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:36.916916 18748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:36.917110 18748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:36.917390 18748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:36.917896 18806 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:05:36.920022 18748 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:05:36.920099 18748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:05:36.920146 18748 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:05:36.920887 18748 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:05:36.921000 18748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:05:36.921016 18806 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Bootstrap starting.
I20260807 15:05:36.928761 18748 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:35843
I20260807 15:05:36.929184 18748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:05:36.929845 18918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:35843 every 8 connection(s)
I20260807 15:05:36.930970 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18748
I20260807 15:05:36.931272 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:45633
--local_ip_for_outbound_sockets=127.1.127.194
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=42641
--webserver_interface=127.1.127.194
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:05:36.933537 18806 log.cc:824] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:36.936254 18806 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:05:36.936620 18806 tablet_bootstrap.cc:492] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Bootstrap complete.
I20260807 15:05:36.937810 18806 ts_tablet_manager.cc:1397] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260807 15:05:36.939116 18806 raft_consensus.cc:348] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:36.939394 18806 raft_consensus.cc:729] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0373ae1509a486d80e6026dff5f83b6, State: Initialized, Role: FOLLOWER
I20260807 15:05:36.939577 18806 consensus_queue.cc:260] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:36.940088 18806 ts_tablet_manager.cc:1428] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260807 15:05:37.020680 18921 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:05:36.929990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:53159)
0807 15:05:36.930191 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:36.930237 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:36.930865 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:37.020425 (+ 89560us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
W20260807 15:05:37.020949 18919 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:40173 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:40173: Client connection negotiation failed: client connection to 127.1.127.254:40173: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:05:37.021327 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.021196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:53707)
0807 15:05:37.021255 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.021306 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:05:37.021821 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.021727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:35713)
0807 15:05:37.021778 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.021804 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 15:05:37.021975 18919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:05:37.037823 18922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:37.038098 18922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:37.038144 18922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:37.039557 18922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:37.039638 18922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:37.039690 18922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:37.041280 18922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:45633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=42641
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:37.041569 18922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:37.041823 18922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:37.047358 18929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:37.047362 18932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:37.047403 18930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:37.047724 18922 server_base.cc:1034] running on GCE node
I20260807 15:05:37.048198 18922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:37.048480 18922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:37.049654 18922 hybrid_clock.cc:648] HybridClock initialized: now 1786115137049630 us; error 50 us; skew 500 ppm
Aug 07 15:05:37 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115137, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:37.052496 18922 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:05:37.052999 18922 webserver.cc:460] Webserver started at http://127.1.127.194:42641/ using document root <none> and password file <none>
I20260807 15:05:37.053239 18922 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:37.053319 18922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:37.054880 18922 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:37.055686 18939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.055888 18922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:37.056003 18922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "840507dc2d394573958948e981180243"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "bd2c8663c7b0b014109b86cd3230c3ee"
server_key_iv: "c210e9cdd5be6e426ef8fd568403d188"
server_key_version: "encryptionkey@0"
I20260807 15:05:37.056432 18922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:37.082955 18922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:37.085565 18922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:37.085848 18922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:37.086210 18922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:37.086557 18922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:37.086614 18922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.086659 18922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:37.086715 18922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.093696 18922 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45633
I20260807 15:05:37.093818 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45633 every 8 connection(s)
I20260807 15:05:37.094179 18922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:05:37.095791 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.095115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:56927)
0807 15:05:37.095322 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.095410 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
W20260807 15:05:37.096057 19061 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:40173 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:40173: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
I20260807 15:05:37.096386 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.096282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:55641)
0807 15:05:37.096333 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.096361 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:05:37.096680 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.096584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:57233)
0807 15:05:37.096632 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.096654 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:05:37.096778 19061 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:05:37.099355 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 18922
I20260807 15:05:37.099742 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:46439
--local_ip_for_outbound_sockets=127.1.127.195
--tserver_master_addrs=127.1.127.254:40173
--webserver_port=45493
--webserver_interface=127.1.127.195
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:37.187933 19064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:37.188213 19064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:37.188256 19064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:37.189462 19064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:37.189544 19064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:37.189592 19064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:37.191273 19064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:46439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=45493
--tserver_master_addrs=127.1.127.254:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:37.191571 19064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:37.191838 19064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:37.197467 19072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:37.197471 19070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:37.197783 19064 server_base.cc:1034] running on GCE node
W20260807 15:05:37.197574 19069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:37.198051 19064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:37.198356 19064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:37.199550 19064 hybrid_clock.cc:648] HybridClock initialized: now 1786115137199515 us; error 49 us; skew 500 ppm
Aug 07 15:05:37 dist-test-slave-8tdl krb5kdc[17558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115137, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:37.202404 19064 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:05:37.203042 19064 webserver.cc:460] Webserver started at http://127.1.127.195:45493/ using document root <none> and password file <none>
I20260807 15:05:37.203310 19064 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:37.203390 19064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:37.205045 19064 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:05:37.205925 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.206212 19064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:37.206386 19064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "b108bec5f0a74a3093282137b10d47d9"
format_stamp: "Formatted at 2026-08-07 15:04:55 on dist-test-slave-8tdl"
server_key: "57371b19703e0a04fec28bfba1224574"
server_key_iv: "af4b9b2a53fc0b7a4b1da4d9f5e75a5e"
server_key_version: "encryptionkey@0"
I20260807 15:05:37.206840 19064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:37.227154 19064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:37.229733 19064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:37.229947 19064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:37.230232 19064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:37.230581 19064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:37.230639 19064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.230676 19064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:37.230737 19064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:37.237838 19064 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:46439
I20260807 15:05:37.237886 19198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:46439 every 8 connection(s)
I20260807 15:05:37.238385 19064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:37.239707 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.239200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:33985)
0807 15:05:37.239421 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.239509 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
W20260807 15:05:37.240258 19199 heartbeater.cc:644] Failed to heartbeat to 127.1.127.254:40173 (0 consecutive failures): Network error: Failed to ping master at 127.1.127.254:40173: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
I20260807 15:05:37.240660 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.240537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:55737)
0807 15:05:37.240599 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.240637 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:05:37.240983 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:37.240881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:34789)
0807 15:05:37.240931 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:37.240959 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:05:37.241094 19199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:05:37.246699 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19064
I20260807 15:05:38.022748 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.022391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:37189)
0807 15:05:38.022574 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:38.022618 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":78,"threads_started":1}
I20260807 15:05:38.097486 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.097232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:39487)
0807 15:05:38.097370 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:38.097416 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260807 15:05:38.241760 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.241499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:57785)
0807 15:05:38.241652 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:38.241699 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260807 15:05:38.646824 19206 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:38.646960 19206 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:38.647287 19206 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
Aug 07 15:05:38 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115136, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.195@KRBTEST.COM
Aug 07 15:05:38 dist-test-slave-8tdl krb5kdc[17558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115136, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.194@KRBTEST.COM
I20260807 15:05:38.661310 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.647584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34181 (local address 127.1.127.194:45633)
0807 15:05:38.647760 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:38.647772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:38.647797 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:05:38.648407 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:38.648410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:38.648480 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:38.648648 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:38.649884 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.651341 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.652981 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.653626 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.653752 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:38.657384 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:38.657389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:38.657389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:38.657401 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:38.659476 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:38.660031 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:38.660033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:38.660033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:38.660131 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:38.660374 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:38.660375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:38.660375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:38.660728 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:38.660831 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:38.661047 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 15:05:38.661189 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":50,"threads_started":1}
I20260807 15:05:38.661293 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.647472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45633 (local address 127.1.127.193:34181)
0807 15:05:38.647658 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:38.647673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:38.647738 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:38.648782 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:38.648786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:38.649428 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:38.649725 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:38.649733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.651521 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.651522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:38.652870 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:38.652874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.653766 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.653768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:38.653948 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:38.654863 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:38.654881 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:38.657242 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:38.659567 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:38.659568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:38.659568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:38.659869 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:38.660212 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:38.660213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:38.660213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:38.660284 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:38.660812 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:38.660813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:38.660875 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 15:05:38.660963 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:05:38.661173 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":131,"thread_start_us":143,"threads_started":1}
I20260807 15:05:38.661311 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.647708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:33089 (local address 127.1.127.195:46439)
0807 15:05:38.647873 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:38.647880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:38.647911 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:05:38.648522 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:38.648525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:38.648569 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:38.648684 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:38.649884 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.651341 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.652985 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.653627 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.653752 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:38.657054 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:38.657061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:38.657061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:38.657077 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:38.659211 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:38.659861 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:38.659865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:38.659866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:38.659983 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:38.660286 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:38.660287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:38.660288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:38.660584 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:38.660685 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:38.661015 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 15:05:38.661158 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260807 15:05:38.661562 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:38.647481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:46439 (local address 127.1.127.193:33089)
0807 15:05:38.647686 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:38.647698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:38.647762 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:38.648782 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:38.648786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:38.649436 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:38.649725 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:38.649733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.651516 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.651517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:38.652870 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:38.652874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:38.653768 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:38.653768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:38.653948 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:38.654952 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:38.654957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:38.656816 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:38.659333 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:38.659347 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:38.659350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:38.659755 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:38.660061 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:38.660062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:38.660063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:38.660200 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:38.660662 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:38.660670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:38.660829 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 15:05:38.660947 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:05:38.661265 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":205,"thread_start_us":96,"threads_started":1}
W20260807 15:05:38.662550 18803 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 840507dc2d394573958948e981180243 (127.1.127.194:45633): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
W20260807 15:05:38.662731 18800 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:38.662798 18800 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0373ae1509a486d80e6026dff5f83b6; no voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9
I20260807 15:05:38.662910 19206 raft_consensus.cc:2726] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:05:39.023878 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:39.023683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:35593)
0807 15:05:39.023776 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:39.023831 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:05:39.098403 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:39.098083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:44483)
0807 15:05:39.098225 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:39.098340 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260807 15:05:39.242762 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:39.242434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:44493)
0807 15:05:39.242604 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:39.242654 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260807 15:05:40.024796 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:40.024503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:43921)
0807 15:05:40.024695 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:40.024746 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260807 15:05:40.099239 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:40.099004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:37065)
0807 15:05:40.099142 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:40.099190 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260807 15:05:40.243738 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:40.243484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:35411)
0807 15:05:40.243645 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:40.243690 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260807 15:05:40.818460 19218 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:40.818570 19218 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:40.818845 19218 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
W20260807 15:05:40.819396 18800 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
W20260807 15:05:40.819459 18803 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 840507dc2d394573958948e981180243 (127.1.127.194:45633): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:40.819486 18800 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0373ae1509a486d80e6026dff5f83b6; no voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9
I20260807 15:05:40.819578 19218 raft_consensus.cc:2726] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:05:41.025913 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:41.025612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:48415)
0807 15:05:41.025799 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:41.025863 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260807 15:05:41.100068 19220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:41.099789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:49547)
0807 15:05:41.099964 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:41.100003 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":62,"threads_started":1}
I20260807 15:05:41.244755 19221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:41.244432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:58217)
0807 15:05:41.244629 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:41.244688 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260807 15:05:42.027092 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:42.026805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:43467)
0807 15:05:42.026991 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:42.027042 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":113,"threads_started":1}
I20260807 15:05:42.100958 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:42.100682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:41639)
0807 15:05:42.100847 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:42.100896 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260807 15:05:42.245921 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:42.245651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:40881)
0807 15:05:42.245801 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:42.245859 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260807 15:05:43.027899 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:43.027629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:46377)
0807 15:05:43.027787 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:43.027848 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260807 15:05:43.101786 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:43.101536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:48071)
0807 15:05:43.101689 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:43.101734 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260807 15:05:43.246894 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:43.246552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:59981)
0807 15:05:43.246793 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:43.246841 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":76,"threads_started":1}
I20260807 15:05:43.482887 19228 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:43.483002 19228 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:43.483309 19228 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
W20260807 15:05:43.483777 18800 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
W20260807 15:05:43.483861 18803 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 840507dc2d394573958948e981180243 (127.1.127.194:45633): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:43.483935 18803 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0373ae1509a486d80e6026dff5f83b6; no voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9
I20260807 15:05:43.484004 19228 raft_consensus.cc:2726] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:05:44.029032 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:44.028643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:42287)
0807 15:05:44.028924 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:44.028978 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260807 15:05:44.102777 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:44.102373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:42291)
0807 15:05:44.102598 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:44.102650 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
I20260807 15:05:44.247740 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:44.247452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:41515)
0807 15:05:44.247643 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:44.247690 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260807 15:05:45.030035 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:45.029743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:51289)
0807 15:05:45.029910 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:45.029964 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260807 15:05:45.103638 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:45.103335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:37377)
0807 15:05:45.103508 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:45.103554 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
I20260807 15:05:45.248773 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:45.248503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:55023)
0807 15:05:45.248674 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:45.248722 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260807 15:05:45.931342 19235 raft_consensus.cc:482] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:45.931450 19235 raft_consensus.cc:504] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840507dc2d394573958948e981180243" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45633 } } peers { permanent_uuid: "b108bec5f0a74a3093282137b10d47d9" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 46439 } } peers { permanent_uuid: "e0373ae1509a486d80e6026dff5f83b6" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 35843 } }
I20260807 15:05:45.931687 19235 leader_election.cc:288] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 840507dc2d394573958948e981180243 (127.1.127.194:45633), b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439)
W20260807 15:05:45.932094 18803 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 840507dc2d394573958948e981180243 (127.1.127.194:45633): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
W20260807 15:05:45.932180 18800 leader_election.cc:341] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b108bec5f0a74a3093282137b10d47d9 (127.1.127.195:46439): Not found: Tablet not found: f1840021b7bf4e7dbb18b5a103e46bb3
I20260807 15:05:45.932245 18800 leader_election.cc:302] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0373ae1509a486d80e6026dff5f83b6; no voters: 840507dc2d394573958948e981180243, b108bec5f0a74a3093282137b10d47d9
I20260807 15:05:45.932340 19235 raft_consensus.cc:2726] T f1840021b7bf4e7dbb18b5a103e46bb3 P e0373ae1509a486d80e6026dff5f83b6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:05:46.031098 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:46.030853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:49287)
0807 15:05:46.030999 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:46.031049 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260807 15:05:46.104436 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:46.104184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:56907)
0807 15:05:46.104343 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:46.104388 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260807 15:05:46.249586 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:46.249355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:51759)
0807 15:05:46.249496 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:46.249539 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260807 15:05:47.031898 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:47.031653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:43989)
0807 15:05:47.031808 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:47.031849 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260807 15:05:47.105307 19240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:47.104971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:36929)
0807 15:05:47.105215 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:47.105256 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":128,"threads_started":1}
I20260807 15:05:47.250624 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:47.250369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:35341)
0807 15:05:47.250525 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:47.250573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260807 15:05:48.032893 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:48.032546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:54235)
0807 15:05:48.032714 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:48.032775 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260807 15:05:48.106083 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:48.105832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:54051)
0807 15:05:48.105988 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:48.106034 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260807 15:05:48.251507 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:48.251239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:36955)
0807 15:05:48.251410 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:48.251454 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":74,"threads_started":1}
I20260807 15:05:49.033704 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:49.033431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:57971)
0807 15:05:49.033592 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:49.033644 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260807 15:05:49.106966 19246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:49.106615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:55577)
0807 15:05:49.106863 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:49.106912 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":142,"threads_started":1}
I20260807 15:05:49.252511 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:49.252175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:45675)
0807 15:05:49.252374 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:49.252429 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260807 15:05:50.034642 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:50.034384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:55505)
0807 15:05:50.034549 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:50.034594 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260807 15:05:50.107729 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:50.107479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:33347)
0807 15:05:50.107635 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:50.107677 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260807 15:05:50.253541 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:50.253260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:43567)
0807 15:05:50.253434 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:50.253486 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260807 15:05:51.035622 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:51.035296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:57447)
0807 15:05:51.035524 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:51.035573 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260807 15:05:51.108721 19252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:51.108387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:53279)
0807 15:05:51.108616 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:51.108667 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260807 15:05:51.254393 19253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:51.254139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:41385)
0807 15:05:51.254298 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:51.254341 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260807 15:05:52.036695 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:52.036397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.193:39733)
0807 15:05:52.036593 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:52.036641 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":134,"threads_started":1}
I20260807 15:05:52.109791 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:52.109526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.194:35993)
0807 15:05:52.109692 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:52.109740 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260807 15:05:52.251365 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18748
I20260807 15:05:52.255398 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:52.255141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:40173 (local address 127.1.127.195:43487)
0807 15:05:52.255296 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:52.255345 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.254:40173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260807 15:05:52.258803 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 18922
I20260807 15:05:52.267017 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19064
2026-08-07T15:05:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58771 ms)
[----------] 30 tests from SecurityITest (249704 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:05:52 dist-test-slave-8tdl krb5kdc[19260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:05:52 dist-test-slave-8tdl krb5kdc[19260](info): set up 2 sockets
Aug 07 15:05:52 dist-test-slave-8tdl krb5kdc[19260](info): commencing operation
krb5kdc: starting...
W20260807 15:05:53.469223 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:05:53 dist-test-slave-8tdl krb5kdc[19260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:05:53.616395 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37757
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:53.712455 19274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:53.712744 19274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:53.712774 19274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:53.714134 19274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:53.714183 19274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:53.714200 19274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:53.714214 19274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:53.714228 19274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:53.716034 19274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:53.716287 19274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:53.716486 19274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:53.722647 19279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:53.722641 19282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:53.722641 19280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:53.723014 19274 server_base.cc:1034] running on GCE node
I20260807 15:05:53.723505 19274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:53.723997 19274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:53.725172 19274 hybrid_clock.cc:648] HybridClock initialized: now 1786115153725143 us; error 48 us; skew 500 ppm
Aug 07 15:05:53 dist-test-slave-8tdl krb5kdc[19260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:53.728823 19274 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:05:53.729471 19274 webserver.cc:460] Webserver started at http://127.1.127.254:41275/ using document root <none> and password file <none>
I20260807 15:05:53.729772 19274 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:53.729844 19274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:53.730003 19274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:53.731503 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "05351763b0914af3bdf6265b63a78442"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "ca24694ce1a55f854840df82c610d63c"
server_key_iv: "12065b14137a204f1da3a607ea0d9009"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.731900 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "05351763b0914af3bdf6265b63a78442"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "ca24694ce1a55f854840df82c610d63c"
server_key_iv: "12065b14137a204f1da3a607ea0d9009"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.733551 19274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:05:53.735073 19290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:53.735276 19274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:05:53.735379 19274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "05351763b0914af3bdf6265b63a78442"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "ca24694ce1a55f854840df82c610d63c"
server_key_iv: "12065b14137a204f1da3a607ea0d9009"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.735455 19274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:53.750226 19274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:53.753016 19274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:53.753173 19274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:53.757782 19274 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:37757
I20260807 15:05:53.757907 19343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:37757 every 8 connection(s)
I20260807 15:05:53.758359 19274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:53.759807 19344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:53.762351 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: Bootstrap starting.
I20260807 15:05:53.763130 19344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:53.763471 19344 log.cc:824] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:53.763669 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19274
I20260807 15:05:53.763804 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:05:53.764078 1535 external_mini_cluster.cc:1428] Setting key e00e4366cb8f75af626af5a8ec3afc16
I20260807 15:05:53.764951 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: No bootstrap required, opened a new log
I20260807 15:05:53.768440 19344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } }
I20260807 15:05:53.768661 19344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:53.768695 19344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05351763b0914af3bdf6265b63a78442, State: Initialized, Role: FOLLOWER
I20260807 15:05:53.768844 19344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } }
I20260807 15:05:53.769140 19344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:53.769210 19344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:53.769270 19344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:53.770100 19344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } }
I20260807 15:05:53.770228 19344 leader_election.cc:302] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05351763b0914af3bdf6265b63a78442; no voters:
I20260807 15:05:53.770437 19344 leader_election.cc:288] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:05:53.770663 19351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:53.770818 19344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:53.770955 19351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [term 1 LEADER]: Becoming Leader. State: Replica: 05351763b0914af3bdf6265b63a78442, State: Running, Role: LEADER
I20260807 15:05:53.771064 19351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } }
I20260807 15:05:53.771487 19350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05351763b0914af3bdf6265b63a78442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } } }
I20260807 15:05:53.771584 19350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:53.771790 19352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05351763b0914af3bdf6265b63a78442. Latest consensus state: current_term: 1 leader_uuid: "05351763b0914af3bdf6265b63a78442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05351763b0914af3bdf6265b63a78442" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37757 } } }
I20260807 15:05:53.771879 19352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442 [sys.catalog]: This master's current role is: LEADER
W20260807 15:05:53.773042 19366 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:05:53.773110 19366 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 15:05:53 dist-test-slave-8tdl krb5kdc[19260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:53.773161 19360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:53.773360 19360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:53.775161 19360 catalog_manager.cc:1347] Generated new cluster ID: 85553e9a5b5a4583af6664bcb0b37be5
I20260807 15:05:53.775223 19360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:53.777906 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:53.764757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50740 (local address 127.1.127.254:37757)
0807 15:05:53.765019 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:53.765031 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:53.765078 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:05:53.765621 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:53.765629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:53.765686 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:53.765813 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:53.766795 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:53.767776 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:53.769186 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:53.769719 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:53.769814 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:53.773723 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:53.773728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:53.773728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:53.773743 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:53.775911 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:53.776368 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:53.776371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:53.776371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:53.776487 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:53.776786 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:53.776787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:53.776787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:53.777126 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:53.777266 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:53.777502 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 15:05:53.777653 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":168,"threads_started":1}
I20260807 15:05:53.782855 19360 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:05:53.784354 19360 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:53.797498 19360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: Generated new TSK 0
I20260807 15:05:53.798151 19360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:05:53.843995 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:53.943312 19372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:53.943563 19372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:53.943593 19372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:53.944969 19372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:53.945040 19372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:53.945073 19372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:53.947379 19372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:53.947722 19372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:53.948058 19372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:53.955089 19378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:53.955068 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:53.955067 19377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:53.955363 19372 server_base.cc:1034] running on GCE node
I20260807 15:05:53.955760 19372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:53.956072 19372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:53.957290 19372 hybrid_clock.cc:648] HybridClock initialized: now 1786115153957277 us; error 33 us; skew 500 ppm
Aug 07 15:05:53 dist-test-slave-8tdl krb5kdc[19260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:53.960393 19372 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:05:53.960836 19372 webserver.cc:460] Webserver started at http://127.1.127.193:34743/ using document root <none> and password file <none>
I20260807 15:05:53.961068 19372 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:53.961114 19372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:53.961208 19372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:53.962282 19372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "ad8a73eb424043c696b98dc12a89405f"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "f3d46e78348722b4ca5eecad2fb302d1"
server_key_iv: "254b937ae19ca442b98771f5b8e0558f"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.962611 19372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "ad8a73eb424043c696b98dc12a89405f"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "f3d46e78348722b4ca5eecad2fb302d1"
server_key_iv: "254b937ae19ca442b98771f5b8e0558f"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.964183 19372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:53.965137 19387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:53.965353 19372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:53.965499 19372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "ad8a73eb424043c696b98dc12a89405f"
format_stamp: "Formatted at 2026-08-07 15:05:53 on dist-test-slave-8tdl"
server_key: "f3d46e78348722b4ca5eecad2fb302d1"
server_key_iv: "254b937ae19ca442b98771f5b8e0558f"
server_key_version: "encryptionkey@0"
I20260807 15:05:53.965576 19372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:54.001595 19372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:54.005118 19372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:54.005268 19372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:54.005511 19372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:54.005934 19372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:54.006047 19372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.006115 19372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:54.006176 19372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.014195 19372 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:39539
I20260807 15:05:54.014331 19506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:39539 every 8 connection(s)
I20260807 15:05:54.014652 19372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:05:54.021600 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19372
I20260807 15:05:54.021760 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:05:54.021970 1535 external_mini_cluster.cc:1428] Setting key d9fe44521ead089ee074c687059928fb
Aug 07 15:05:54 dist-test-slave-8tdl krb5kdc[19260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:54.027985 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.015628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34463 (local address 127.1.127.254:37757)
0807 15:05:54.015702 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.015705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.015925 (+ 220us) server_negotiation.cc:407] Connection header received
0807 15:05:54.016523 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.016526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.016546 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.016661 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.018593 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.019365 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.020647 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.020876 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.020952 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.023991 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.023992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.023993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:54.024011 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.026264 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.026812 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.026813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.026814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.026882 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.027155 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.027157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.027157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.027294 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.027389 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.027698 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.027821 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:05:54.028142 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.015542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37757 (local address 127.1.127.193:34463)
0807 15:05:54.015737 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.015788 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.016331 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.016806 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.016811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.017809 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:54.018166 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.018174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.019529 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.019530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.020524 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.020527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.021028 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.021029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.021171 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.021915 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:54.021925 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:54.023859 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:54.026391 (+ 2532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.026402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.026405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.026669 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.026960 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.026961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.026962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.027085 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.027381 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:54.027389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:54.027552 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.027666 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.027873 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260807 15:05:54.028858 19507 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37757
I20260807 15:05:54.028973 19507 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:54.029212 19507 heartbeater.cc:511] Master 127.1.127.254:37757 requested a full tablet report, sending...
I20260807 15:05:54.029850 19307 ts_manager.cc:194] Registered new tserver with Master: ad8a73eb424043c696b98dc12a89405f (127.1.127.193:39539)
I20260807 15:05:54.030649 19307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:34463
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:05:54.059798 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:54.147837 19512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:54.148074 19512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:54.148101 19512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:54.149364 19512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:54.149416 19512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:54.149446 19512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:54.151172 19512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:54.151415 19512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:54.151607 19512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:05:54.157883 19512 server_base.cc:1034] running on GCE node
W20260807 15:05:54.157861 19517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.157809 19518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.157859 19520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.158838 19512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:54.159134 19512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:54.160312 19512 hybrid_clock.cc:648] HybridClock initialized: now 1786115154160299 us; error 34 us; skew 500 ppm
Aug 07 15:05:54 dist-test-slave-8tdl krb5kdc[19260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:54.163384 19512 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:05:54.163831 19512 webserver.cc:460] Webserver started at http://127.1.127.194:32919/ using document root <none> and password file <none>
I20260807 15:05:54.164106 19512 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:54.164183 19512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:54.164338 19512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:54.165472 19512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "d7a913b62fa34d419c7e27166f448d68"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "24ee4841d76f55d27bdcfd90ecba3559"
server_key_iv: "0d1d40981ee2432befd6923c974e095a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.165865 19512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "d7a913b62fa34d419c7e27166f448d68"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "24ee4841d76f55d27bdcfd90ecba3559"
server_key_iv: "0d1d40981ee2432befd6923c974e095a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.167461 19512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:05:54.168471 19527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.168816 19512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:54.168948 19512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "d7a913b62fa34d419c7e27166f448d68"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "24ee4841d76f55d27bdcfd90ecba3559"
server_key_iv: "0d1d40981ee2432befd6923c974e095a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.169041 19512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:54.184826 19512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:54.187621 19512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:54.187767 19512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:54.188103 19512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:54.188493 19512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:54.188530 19512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.188555 19512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:54.188567 19512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.195293 19512 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:40101
I20260807 15:05:54.195345 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:40101 every 8 connection(s)
I20260807 15:05:54.195724 19512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:05:54.196679 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19512
I20260807 15:05:54.196798 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:05:54.197042 1535 external_mini_cluster.cc:1428] Setting key 0ec4626bfd457ff851f6d7bac6901f73
Aug 07 15:05:54 dist-test-slave-8tdl krb5kdc[19260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
I20260807 15:05:54.208518 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.196601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:44525 (local address 127.1.127.254:37757)
0807 15:05:54.196681 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.196683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.196927 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:05:54.197602 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.197603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.197613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.197678 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.198843 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.199622 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.201294 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.201583 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.201721 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.204894 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.204896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.204896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:54.204906 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.206645 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.207287 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.207288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.207289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.207367 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.207682 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.207684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.207684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.207824 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.207913 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.208326 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.208429 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:05:54.208752 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.196582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37757 (local address 127.1.127.194:44525)
0807 15:05:54.196795 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.196843 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.197507 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.197771 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.197776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.198450 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:54.198727 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.198735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.199749 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.199750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.201170 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.201173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.201743 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.201745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.201984 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.202666 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:54.202722 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:54.204730 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:54.206803 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.206814 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.206817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.207173 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.207454 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.207454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.207455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.207602 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.207903 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:54.207920 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:54.208126 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.208266 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.208469 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":125,"threads_started":1}
I20260807 15:05:54.209641 19644 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37757
I20260807 15:05:54.210131 19644 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:54.210419 19644 heartbeater.cc:511] Master 127.1.127.254:37757 requested a full tablet report, sending...
I20260807 15:05:54.211228 19307 ts_manager.cc:194] Registered new tserver with Master: d7a913b62fa34d419c7e27166f448d68 (127.1.127.194:40101)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
I20260807 15:05:54.211809 19307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:44525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:05:54.234406 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:54.332247 19649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:54.332496 19649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:54.332528 19649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:54.333842 19649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:54.333890 19649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:54.333921 19649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:54.335786 19649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:54.336086 19649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:54.336344 19649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:54.342618 19659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.342602 19655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.342599 19649 server_base.cc:1034] running on GCE node
W20260807 15:05:54.342593 19656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.343112 19649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:54.343415 19649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:54.344610 19649 hybrid_clock.cc:648] HybridClock initialized: now 1786115154344585 us; error 41 us; skew 500 ppm
Aug 07 15:05:54 dist-test-slave-8tdl krb5kdc[19260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:54.347456 19649 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:05:54.347973 19649 webserver.cc:460] Webserver started at http://127.1.127.195:46219/ using document root <none> and password file <none>
I20260807 15:05:54.348233 19649 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:54.348306 19649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:54.348456 19649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:54.349567 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "e1758b5545394ba8adddf1961dee7045"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "988ebd1d0dce77ddb1bfc92c9467bdad"
server_key_iv: "4ab9fdd8404a0acae40757269d8970c4"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.349951 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "e1758b5545394ba8adddf1961dee7045"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "988ebd1d0dce77ddb1bfc92c9467bdad"
server_key_iv: "4ab9fdd8404a0acae40757269d8970c4"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.351555 19649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:05:54.352384 19667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.352767 19649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:54.352883 19649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "e1758b5545394ba8adddf1961dee7045"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "988ebd1d0dce77ddb1bfc92c9467bdad"
server_key_iv: "4ab9fdd8404a0acae40757269d8970c4"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.352968 19649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:54.368616 19649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:54.370913 19649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:54.371294 19649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:54.371618 19649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:54.371999 19649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:54.372053 19649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.372097 19649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:54.372120 19649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.378559 19649 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:41971
I20260807 15:05:54.378607 19783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:41971 every 8 connection(s)
I20260807 15:05:54.379014 19649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:54.381184 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19649
I20260807 15:05:54.381310 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:05:54.381525 1535 external_mini_cluster.cc:1428] Setting key b2a4973727e45df79b95e306be4d9787
Aug 07 15:05:54 dist-test-slave-8tdl krb5kdc[19260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:54.390722 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.379870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:38949 (local address 127.1.127.254:37757)
0807 15:05:54.379968 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.379970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.380186 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:05:54.380714 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.380716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.380727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.380779 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.382293 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.383009 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.384183 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.384386 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.384451 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.387271 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.387273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.387274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:54.387283 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.388808 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.389426 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.389428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.389428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.389513 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.389841 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.389842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.389842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.390039 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.390131 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.390472 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.390573 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:05:54.390992 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.379806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37757 (local address 127.1.127.195:38949)
0807 15:05:54.380036 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.380089 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.380625 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.380844 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.380847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.381769 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:54.382096 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.382103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.383128 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.383129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.384078 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.384081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.384473 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.384474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.384590 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.385360 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:54.385369 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:54.387129 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:54.388929 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.388939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.388942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.389213 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.389600 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:54.389601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:54.389602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:54.389750 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:54.390133 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:54.390142 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:54.390309 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.390449 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.390663 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
I20260807 15:05:54.391639 19784 heartbeater.cc:347] Connected to a master server at 127.1.127.254:37757
I20260807 15:05:54.391741 19784 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:54.391911 19784 heartbeater.cc:511] Master 127.1.127.254:37757 requested a full tablet report, sending...
I20260807 15:05:54.392370 19307 ts_manager.cc:194] Registered new tserver with Master: e1758b5545394ba8adddf1961dee7045 (127.1.127.195:41971)
I20260807 15:05:54.392771 19307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:38949
I20260807 15:05:54.393875 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:05:54.402477 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.395352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.1.127.254:37757)
0807 15:05:54.395417 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.395418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.395548 (+ 130us) server_negotiation.cc:407] Connection header received
0807 15:05:54.395602 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.395603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.395613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.395662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.396332 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.396843 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.397697 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.397898 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.398016 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.399653 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.399654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.399655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:54.399662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.401255 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.401633 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.401633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.401634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.401689 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:54.401934 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:54.401935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:54.401935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:54.402091 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.402172 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.402300 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.402371 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:05:54.404596 19307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:05:54.406423 19307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:05:54.414640 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.409615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41971 (local address 127.0.0.1:40240)
0807 15:05:54.410160 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.410173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.410238 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.410751 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.410752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.410754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:54.410797 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.410799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.411784 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.411785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.413301 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.413305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414220 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.414331 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414351 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.414439 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.414516 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":249,"thread_start_us":66,"threads_started":1}
I20260807 15:05:54.414983 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.409811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.1.127.193:39539)
0807 15:05:54.410138 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.410145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.410157 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:05:54.410203 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.410206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.410223 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.410341 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:54.410607 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.411883 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.413426 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414334 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414715 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414784 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.414832 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.414884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":200,"threads_started":1}
I20260807 15:05:54.414801 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.409963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.1.127.194:40101)
0807 15:05:54.410152 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.410156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.410166 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:54.410203 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.410206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.410223 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.410341 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:54.410661 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.411883 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.413452 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414236 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414387 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414469 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.414623 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.414707 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260807 15:05:54.415185 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.409623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39539 (local address 127.0.0.1:53110)
0807 15:05:54.409918 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.409927 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.409972 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.410449 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.410450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.410452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:54.410522 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.410524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.412017 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.412018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.413321 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.413323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414434 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.414511 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414515 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.414555 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.414640 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":41,"thread_start_us":108,"threads_started":1}
I20260807 15:05:54.415335 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.409615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:40101 (local address 127.0.0.1:49302)
0807 15:05:54.409816 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.409871 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.409958 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.410380 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.410383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.410386 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:54.410491 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.410493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.412067 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.412068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.413327 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.413331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414425 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.414516 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414521 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.414584 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.414631 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260807 15:05:54.415623 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.410158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.1.127.195:41971)
0807 15:05:54.410481 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.410487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.410498 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:05:54.410538 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.410540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.410570 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.410678 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:54.410931 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.411614 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.413425 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.414125 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.414217 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.414295 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.415441 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.415527 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":116,"threads_started":1}
I20260807 15:05:54.416215 19432 tablet_service.cc:1467] Processing CreateTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 (DEFAULT_TABLE table=test-table [id=b06b6e7ab9424001895c76d5e374fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:54.416420 19717 tablet_service.cc:1467] Processing CreateTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 (DEFAULT_TABLE table=test-table [id=b06b6e7ab9424001895c76d5e374fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:54.416576 19432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34971973b3144540b1de7ce36fbcfbc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:54.416709 19717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34971973b3144540b1de7ce36fbcfbc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:54.418833 19577 tablet_service.cc:1467] Processing CreateTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 (DEFAULT_TABLE table=test-table [id=b06b6e7ab9424001895c76d5e374fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:54.419474 19577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34971973b3144540b1de7ce36fbcfbc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:54.420683 19802 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Bootstrap starting.
I20260807 15:05:54.421185 19803 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Bootstrap starting.
I20260807 15:05:54.421537 19802 tablet_bootstrap.cc:654] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:54.422171 19803 tablet_bootstrap.cc:654] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:54.422254 19802 log.cc:824] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:54.422818 19803 log.cc:824] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:54.423415 19802 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: No bootstrap required, opened a new log
I20260807 15:05:54.423481 19802 ts_tablet_manager.cc:1397] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:05:54.423918 19807 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Bootstrap starting.
I20260807 15:05:54.424475 19803 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: No bootstrap required, opened a new log
I20260807 15:05:54.424579 19803 ts_tablet_manager.cc:1397] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:05:54.424645 19807 tablet_bootstrap.cc:654] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:54.425297 19807 log.cc:824] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:54.425860 19802 raft_consensus.cc:348] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.426090 19802 raft_consensus.cc:374] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:54.426139 19802 raft_consensus.cc:729] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a73eb424043c696b98dc12a89405f, State: Initialized, Role: FOLLOWER
I20260807 15:05:54.426235 19807 tablet_bootstrap.cc:492] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: No bootstrap required, opened a new log
I20260807 15:05:54.426302 19807 ts_tablet_manager.cc:1397] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:05:54.426312 19802 consensus_queue.cc:260] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.426666 19802 ts_tablet_manager.cc:1428] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:05:54.426719 19507 heartbeater.cc:503] Master 127.1.127.254:37757 was elected leader, sending a full tablet report...
I20260807 15:05:54.426628 19803 raft_consensus.cc:348] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.427211 19803 raft_consensus.cc:374] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:54.427263 19803 raft_consensus.cc:729] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1758b5545394ba8adddf1961dee7045, State: Initialized, Role: FOLLOWER
I20260807 15:05:54.427443 19803 consensus_queue.cc:260] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.427775 19784 heartbeater.cc:503] Master 127.1.127.254:37757 was elected leader, sending a full tablet report...
I20260807 15:05:54.427896 19803 ts_tablet_manager.cc:1428] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:05:54.429034 19807 raft_consensus.cc:348] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.429234 19807 raft_consensus.cc:374] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:54.429283 19807 raft_consensus.cc:729] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a913b62fa34d419c7e27166f448d68, State: Initialized, Role: FOLLOWER
I20260807 15:05:54.429404 19807 consensus_queue.cc:260] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.429715 19807 ts_tablet_manager.cc:1428] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:05:54.429814 19644 heartbeater.cc:503] Master 127.1.127.254:37757 was elected leader, sending a full tablet report...
W20260807 15:05:54.446424 19645 tablet.cc:2369] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:05:54.515695 19508 tablet.cc:2369] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:54.541189 19812 raft_consensus.cc:482] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:54.541378 19812 raft_consensus.cc:504] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.541816 19812 leader_election.cc:288] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad8a73eb424043c696b98dc12a89405f (127.1.127.193:39539), e1758b5545394ba8adddf1961dee7045 (127.1.127.195:41971)
I20260807 15:05:54.545128 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.542115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39539 (local address 127.1.127.194:57041)
0807 15:05:54.542182 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.542194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.542243 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.542415 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.542416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.542418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:54.542510 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.542512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.543250 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.543251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.544097 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.544099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.544769 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.544860 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.544866 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.545002 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.545043 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:05:54.545156 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.542182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57041 (local address 127.1.127.193:39539)
0807 15:05:54.542249 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.542251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.542268 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:05:54.542304 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.542306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.542318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.542397 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:54.542596 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.543168 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544224 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.544677 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544738 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.544747 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.545026 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.545074 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:05:54.545250 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.542126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41971 (local address 127.1.127.194:52193)
0807 15:05:54.542339 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.542349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.542400 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.542584 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.542585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.542586 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:54.542665 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.542666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.543384 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.543385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.544215 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.544218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.544962 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.545055 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.545059 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.545122 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.545159 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":3,"thread_start_us":128,"threads_started":1}
I20260807 15:05:54.545265 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.542296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:52193 (local address 127.1.127.195:41971)
0807 15:05:54.542422 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.542424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.542434 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:54.542459 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.542461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.542472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.542544 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:54.542776 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.543270 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544327 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.544808 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.544892 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.544900 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.545142 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.545193 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 15:05:54.545576 19737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34971973b3144540b1de7ce36fbcfbc5" candidate_uuid: "d7a913b62fa34d419c7e27166f448d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1758b5545394ba8adddf1961dee7045" is_pre_election: true
I20260807 15:05:54.545584 19459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34971973b3144540b1de7ce36fbcfbc5" candidate_uuid: "d7a913b62fa34d419c7e27166f448d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a73eb424043c696b98dc12a89405f" is_pre_election: true
I20260807 15:05:54.545740 19459 raft_consensus.cc:2454] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a913b62fa34d419c7e27166f448d68 in term 0.
I20260807 15:05:54.545740 19737 raft_consensus.cc:2454] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a913b62fa34d419c7e27166f448d68 in term 0.
I20260807 15:05:54.546031 19530 leader_election.cc:302] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7a913b62fa34d419c7e27166f448d68, e1758b5545394ba8adddf1961dee7045; no voters:
I20260807 15:05:54.546187 19812 raft_consensus.cc:2781] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:05:54.546245 19812 raft_consensus.cc:482] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:05:54.546273 19812 raft_consensus.cc:3037] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:54.547082 19812 raft_consensus.cc:504] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.547250 19812 leader_election.cc:288] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [CANDIDATE]: Term 1 election: Requested vote from peers ad8a73eb424043c696b98dc12a89405f (127.1.127.193:39539), e1758b5545394ba8adddf1961dee7045 (127.1.127.195:41971)
I20260807 15:05:54.547416 19459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34971973b3144540b1de7ce36fbcfbc5" candidate_uuid: "d7a913b62fa34d419c7e27166f448d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a73eb424043c696b98dc12a89405f"
I20260807 15:05:54.547462 19737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34971973b3144540b1de7ce36fbcfbc5" candidate_uuid: "d7a913b62fa34d419c7e27166f448d68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1758b5545394ba8adddf1961dee7045"
I20260807 15:05:54.547500 19459 raft_consensus.cc:3037] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:54.547576 19737 raft_consensus.cc:3037] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:54.548266 19459 raft_consensus.cc:2454] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a913b62fa34d419c7e27166f448d68 in term 1.
I20260807 15:05:54.548457 19531 leader_election.cc:302] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad8a73eb424043c696b98dc12a89405f, d7a913b62fa34d419c7e27166f448d68; no voters:
I20260807 15:05:54.548473 19737 raft_consensus.cc:2454] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a913b62fa34d419c7e27166f448d68 in term 1.
I20260807 15:05:54.548605 19812 raft_consensus.cc:2781] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:54.548786 19812 raft_consensus.cc:686] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 1 LEADER]: Becoming Leader. State: Replica: d7a913b62fa34d419c7e27166f448d68, State: Running, Role: LEADER
I20260807 15:05:54.548990 19812 consensus_queue.cc:237] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } }
I20260807 15:05:54.549909 19307 catalog_manager.cc:5495] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a913b62fa34d419c7e27166f448d68 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "d7a913b62fa34d419c7e27166f448d68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7a913b62fa34d419c7e27166f448d68" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 40101 } health_report { overall_health: HEALTHY } } }
I20260807 15:05:54.563158 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.560470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.1.127.194:40101)
0807 15:05:54.560546 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.560548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.560555 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:05:54.560591 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.560592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.560611 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.560698 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:54.560918 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.561417 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.562257 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.562465 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.562548 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.562780 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:54.562897 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:05:54.563006 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.563034 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.563057 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:05:54.565456 19459 raft_consensus.cc:1261] T 34971973b3144540b1de7ce36fbcfbc5 P ad8a73eb424043c696b98dc12a89405f [term 1 FOLLOWER]: Refusing update from remote peer d7a913b62fa34d419c7e27166f448d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:54.565456 19737 raft_consensus.cc:1261] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 1 FOLLOWER]: Refusing update from remote peer d7a913b62fa34d419c7e27166f448d68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:54.566009 19814 consensus_queue.cc:1035] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1758b5545394ba8adddf1961dee7045" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:54.566136 19812 consensus_queue.cc:1035] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad8a73eb424043c696b98dc12a89405f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:54.569530 19819 mvcc.cc:205] Tried to move back new op lower bound from 7315927673096564736 to 7315927673033310208. Current Snapshot: MvccSnapshot[applied={T|T < 7315927673096564736}]
I20260807 15:05:54.570235 19818 mvcc.cc:205] Tried to move back new op lower bound from 7315927673096564736 to 7315927673033310208. Current Snapshot: MvccSnapshot[applied={T|T < 7315927673096564736}]
I20260807 15:05:54.571899 19306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:54.572021 19306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:54.573037 19306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05351763b0914af3bdf6265b63a78442: Sending DeleteTablet for 3 replicas of tablet 34971973b3144540b1de7ce36fbcfbc5
I20260807 15:05:54.573537 19432 tablet_service.cc:1514] Processing DeleteTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:53110
I20260807 15:05:54.573521 19577 tablet_service.cc:1514] Processing DeleteTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:49302
I20260807 15:05:54.573796 19717 tablet_service.cc:1514] Processing DeleteTablet for tablet 34971973b3144540b1de7ce36fbcfbc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:54 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:40240
I20260807 15:05:54.574061 19830 tablet_replica.cc:330] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: stopping tablet replica
I20260807 15:05:54.574112 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19372
I20260807 15:05:54.574111 19829 tablet_replica.cc:330] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: stopping tablet replica
I20260807 15:05:54.574237 19829 raft_consensus.cc:2229] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:05:54.574347 19829 raft_consensus.cc:2258] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:54.575325 19829 ts_tablet_manager.cc:1905] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:54.577792 19829 ts_tablet_manager.cc:1918] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:54.578088 19829 log.cc:1196] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/34971973b3144540b1de7ce36fbcfbc5
I20260807 15:05:54.578537 19829 ts_tablet_manager.cc:1939] T 34971973b3144540b1de7ce36fbcfbc5 P d7a913b62fa34d419c7e27166f448d68: Deleting consensus metadata
I20260807 15:05:54.581670 19830 raft_consensus.cc:2229] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:54.581878 19830 raft_consensus.cc:2258] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:54.582592 19830 ts_tablet_manager.cc:1905] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:54.584863 19830 ts_tablet_manager.cc:1918] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:54.584954 19830 log.cc:1196] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/34971973b3144540b1de7ce36fbcfbc5
I20260807 15:05:54.585245 19830 ts_tablet_manager.cc:1939] T 34971973b3144540b1de7ce36fbcfbc5 P e1758b5545394ba8adddf1961dee7045: Deleting consensus metadata
I20260807 15:05:54.585891 19293 catalog_manager.cc:4841] TS e1758b5545394ba8adddf1961dee7045 (127.1.127.195:41971): tablet 34971973b3144540b1de7ce36fbcfbc5 (table test-table [id=b06b6e7ab9424001895c76d5e374fc49]) successfully deleted
I20260807 15:05:54.586526 19292 catalog_manager.cc:4841] TS d7a913b62fa34d419c7e27166f448d68 (127.1.127.194:40101): tablet 34971973b3144540b1de7ce36fbcfbc5 (table test-table [id=b06b6e7ab9424001895c76d5e374fc49]) successfully deleted
W20260807 15:05:54.587471 19294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:39539 (error 108)
I20260807 15:05:54.588145 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.587977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:39539 (local address 127.0.0.1:53114)
0807 15:05:54.588053 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.588102 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:39539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 15:05:54.588179 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19512
W20260807 15:05:54.588284 19294 catalog_manager.cc:4584] TS ad8a73eb424043c696b98dc12a89405f (127.1.127.193:39539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34971973b3144540b1de7ce36fbcfbc5: Network error: Client connection negotiation failed: client connection to 127.1.127.193:39539: connect: Connection refused (error 111)
I20260807 15:05:54.597664 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19649
I20260807 15:05:54.603652 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19274
2026-08-07T15:05:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2336 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T15:05:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:05:54Z Disabled control of system clock
I20260807 15:05:54.636917 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46391
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:46391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:54.725903 19835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:54.726150 19835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:54.726172 19835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:54.727530 19835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:54.727582 19835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:54.727596 19835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:54.727609 19835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:54.727622 19835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:54.729112 19835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:46391
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:54.729418 19835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:54.729802 19835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:54.735229 19840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.735232 19841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.735311 19835 server_base.cc:1034] running on GCE node
W20260807 15:05:54.735229 19843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.735662 19835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:54.736064 19835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:54.737267 19835 hybrid_clock.cc:648] HybridClock initialized: now 1786115154737218 us; error 64 us; skew 500 ppm
I20260807 15:05:54.738963 19835 webserver.cc:460] Webserver started at http://127.1.127.254:46507/ using document root <none> and password file <none>
I20260807 15:05:54.739390 19835 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:54.739452 19835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:54.739620 19835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:54.741070 19835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "e4ef5a86f93347bcbcf44047f057d8cd"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "3cb8b56efb98333dd7ef0238d1ade7cb"
server_key_iv: "02ccd7f5f0ec52f652822aa3bb8029c7"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.741756 19835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "e4ef5a86f93347bcbcf44047f057d8cd"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "3cb8b56efb98333dd7ef0238d1ade7cb"
server_key_iv: "02ccd7f5f0ec52f652822aa3bb8029c7"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.743625 19835 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:05:54.744884 19850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.745179 19835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:54.745307 19835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e4ef5a86f93347bcbcf44047f057d8cd"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "3cb8b56efb98333dd7ef0238d1ade7cb"
server_key_iv: "02ccd7f5f0ec52f652822aa3bb8029c7"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.745415 19835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:54.767405 19835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:54.767814 19835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:54.767983 19835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:54.772650 19835 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:46391
I20260807 15:05:54.772691 19904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:46391 every 8 connection(s)
I20260807 15:05:54.773078 19835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:54.774055 19905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:54.776576 19905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: Bootstrap starting.
I20260807 15:05:54.777137 19905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:54.777408 19905 log.cc:824] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:54.778116 19905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: No bootstrap required, opened a new log
I20260807 15:05:54.779925 19905 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } }
I20260807 15:05:54.780109 19905 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:54.780134 19905 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ef5a86f93347bcbcf44047f057d8cd, State: Initialized, Role: FOLLOWER
I20260807 15:05:54.780293 19905 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } }
I20260807 15:05:54.780380 19905 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:54.780404 19905 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:54.780435 19905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:54.781219 19905 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } }
I20260807 15:05:54.781354 19905 leader_election.cc:302] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4ef5a86f93347bcbcf44047f057d8cd; no voters:
I20260807 15:05:54.781543 19905 leader_election.cc:288] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:05:54.781670 19908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:54.781872 19908 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [term 1 LEADER]: Becoming Leader. State: Replica: e4ef5a86f93347bcbcf44047f057d8cd, State: Running, Role: LEADER
I20260807 15:05:54.781992 19908 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } }
I20260807 15:05:54.782078 19905 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:54.782491 19909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } } }
I20260807 15:05:54.782639 19910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4ef5a86f93347bcbcf44047f057d8cd. Latest consensus state: current_term: 1 leader_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef5a86f93347bcbcf44047f057d8cd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46391 } } }
I20260807 15:05:54.782738 19909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:54.782847 19910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:54.782909 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19835
I20260807 15:05:54.783028 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:05:54.783142 19916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:54.783250 1535 external_mini_cluster.cc:1428] Setting key 16929f44d1b21917fdc52812fb87cde1
I20260807 15:05:54.783823 19916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:54.788995 19916 catalog_manager.cc:1347] Generated new cluster ID: 36cf85d5850940b7a158e2f27b18ab49
I20260807 15:05:54.789083 19916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:54.790010 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.784038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.1.127.254:46391)
0807 15:05:54.784289 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.784301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.784626 (+ 325us) server_negotiation.cc:407] Connection header received
0807 15:05:54.784809 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.784820 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.784915 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.785089 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.785342 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.786400 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.787390 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.787929 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.788094 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.788996 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.789000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.789001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:54.789271 (+ 270us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.789324 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:54.789341 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.789416 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.789619 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.789726 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260807 15:05:54.797111 19916 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:05:54.797693 19916 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:54.804605 19916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: Generated new TSK 0
I20260807 15:05:54.804821 19916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:05:54.811041 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--builtin_ntp_servers=127.1.127.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:54.901521 19930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:54.901768 19930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:54.901796 19930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:54.903250 19930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:54.903312 19930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:54.903345 19930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:54.904911 19930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:54.905216 19930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:54.905567 19930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:54.912571 19937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.912591 19939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:54.912639 19936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:54.912886 19930 server_base.cc:1034] running on GCE node
I20260807 15:05:54.913584 19930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:54.913949 19930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:54.915114 19930 hybrid_clock.cc:648] HybridClock initialized: now 1786115154915093 us; error 40 us; skew 500 ppm
I20260807 15:05:54.922254 19930 webserver.cc:460] Webserver started at http://127.1.127.193:33465/ using document root <none> and password file <none>
I20260807 15:05:54.922752 19930 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:54.922835 19930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:54.923009 19930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:54.924211 19930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "b0c7dcf9975941869f11c1cffab5359d"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "26129f4cee8b6a98d98b68dff759be9c"
server_key_iv: "a83353dfe49a51f05569657674c0703a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.924567 19930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "b0c7dcf9975941869f11c1cffab5359d"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "26129f4cee8b6a98d98b68dff759be9c"
server_key_iv: "a83353dfe49a51f05569657674c0703a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.926297 19930 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:05:54.927392 19945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.927767 19930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:54.927897 19930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "b0c7dcf9975941869f11c1cffab5359d"
format_stamp: "Formatted at 2026-08-07 15:05:54 on dist-test-slave-8tdl"
server_key: "26129f4cee8b6a98d98b68dff759be9c"
server_key_iv: "a83353dfe49a51f05569657674c0703a"
server_key_version: "encryptionkey@0"
I20260807 15:05:54.928023 19930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:54.951330 19930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:54.951728 19930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:54.951906 19930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:54.952211 19930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:54.952595 19930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:54.952657 19930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.952699 19930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:54.952729 19930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:54.959436 19930 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:40313
I20260807 15:05:54.959501 20062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:40313 every 8 connection(s)
I20260807 15:05:54.959911 19930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:05:54.966297 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.960823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:58067 (local address 127.1.127.254:46391)
0807 15:05:54.960945 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:54.960948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:54.961242 (+ 294us) server_negotiation.cc:407] Connection header received
0807 15:05:54.961915 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:54.961916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:54.961938 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:54.962008 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:54.962846 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.963413 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.964548 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.964792 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.964923 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.965977 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:54.965979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:54.965979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:54.965985 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:54.966002 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:54.966009 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:54.966062 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:54.966173 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:05:54.966210 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 15:05:54.966553 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:54.960743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46391 (local address 127.1.127.193:58067)
0807 15:05:54.961085 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:54.961137 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:54.961785 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:54.962085 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:54.962090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:54.962101 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:54.962735 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.962742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.963533 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.963534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.964436 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:54.964439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:54.964942 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:54.964944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:54.965106 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:54.965764 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:54.965765 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:54.965797 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:54.965823 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:54.966063 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:54.966071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:54.966074 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:54.966164 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:05:54.966281 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":92,"threads_started":1}
I20260807 15:05:54.967280 20063 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46391
I20260807 15:05:54.967396 20063 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:54.967613 20063 heartbeater.cc:511] Master 127.1.127.254:46391 requested a full tablet report, sending...
I20260807 15:05:54.967993 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 19930
I20260807 15:05:54.968219 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:05:54.968268 19867 ts_manager.cc:194] Registered new tserver with Master: b0c7dcf9975941869f11c1cffab5359d (127.1.127.193:40313)
I20260807 15:05:54.968585 1535 external_mini_cluster.cc:1428] Setting key 0c38b566c4a140b2f3a142f5dd7394b6
I20260807 15:05:54.969269 19867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.193:58067
I20260807 15:05:54.969833 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--builtin_ntp_servers=127.1.127.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:55.066570 20067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:55.067001 20067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:55.067044 20067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:55.069255 20067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:55.069341 20067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:55.069413 20067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:55.071734 20067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:55.072086 20067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:55.072530 20067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:55.079963 20076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:55.080003 20074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:55.080080 20073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:55.080605 20067 server_base.cc:1034] running on GCE node
I20260807 15:05:55.080905 20067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:55.081310 20067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:55.082477 20067 hybrid_clock.cc:648] HybridClock initialized: now 1786115155082423 us; error 72 us; skew 500 ppm
I20260807 15:05:55.084192 20067 webserver.cc:460] Webserver started at http://127.1.127.194:40961/ using document root <none> and password file <none>
I20260807 15:05:55.084529 20067 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:55.084614 20067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:55.084861 20067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:55.086760 20067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "32e8dc734e5848888289d0bdb43530ae"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "2c0715c488fc9522aaa14c56cf75822b"
server_key_iv: "8426957e5e6caba1190f1e0f6248155f"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.087263 20067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "32e8dc734e5848888289d0bdb43530ae"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "2c0715c488fc9522aaa14c56cf75822b"
server_key_iv: "8426957e5e6caba1190f1e0f6248155f"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.089303 20067 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:05:55.090373 20082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.090560 20067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:55.090667 20067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "32e8dc734e5848888289d0bdb43530ae"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "2c0715c488fc9522aaa14c56cf75822b"
server_key_iv: "8426957e5e6caba1190f1e0f6248155f"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.090786 20067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:55.104091 20067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:55.104575 20067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:55.104781 20067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:55.105146 20067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:55.105638 20067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:55.105713 20067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.105762 20067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:55.105801 20067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.113731 20067 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:45659
I20260807 15:05:55.113771 20197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:45659 every 8 connection(s)
I20260807 15:05:55.114151 20067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:05:55.116241 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20067
I20260807 15:05:55.116369 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:05:55.116571 1535 external_mini_cluster.cc:1428] Setting key 062d3feea2d6bf08808b667ce55fa801
I20260807 15:05:55.117653 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--builtin_ntp_servers=127.1.127.212:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:05:55.120675 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.115034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:45321 (local address 127.1.127.254:46391)
0807 15:05:55.115111 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.115113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.115527 (+ 414us) server_negotiation.cc:407] Connection header received
0807 15:05:55.116037 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.116039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.116053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.116138 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:55.117105 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.117709 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.119030 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.119226 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.119310 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.120339 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:55.120341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:55.120342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:55.120348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:55.120368 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:55.120378 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:55.120444 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.120553 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.120589 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:05:55.120921 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.115112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46391 (local address 127.1.127.194:45321)
0807 15:05:55.115343 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.115382 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.115869 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.116210 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.116213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.116222 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:55.117005 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.117012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.117816 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.117817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.118651 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.118653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.119336 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.119338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.119502 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.120149 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:55.120150 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:55.120182 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:55.120210 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:55.120447 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:55.120456 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:55.120459 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.120535 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.120661 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260807 15:05:55.121531 20198 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46391
I20260807 15:05:55.121666 20198 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:55.121886 20198 heartbeater.cc:511] Master 127.1.127.254:46391 requested a full tablet report, sending...
I20260807 15:05:55.122448 19867 ts_manager.cc:194] Registered new tserver with Master: 32e8dc734e5848888289d0bdb43530ae (127.1.127.194:45659)
I20260807 15:05:55.123138 19867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.194:45321
W20260807 15:05:55.216324 20201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:55.216583 20201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:55.216611 20201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:55.218004 20201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:55.218070 20201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:55.218101 20201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:55.219789 20201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:55.220134 20201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:55.220510 20201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:55.227613 20207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:55.227931 20208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:55.228063 20201 server_base.cc:1034] running on GCE node
W20260807 15:05:55.228173 20210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:55.228618 20201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:55.228987 20201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:55.230185 20201 hybrid_clock.cc:648] HybridClock initialized: now 1786115155230150 us; error 45 us; skew 500 ppm
I20260807 15:05:55.232061 20201 webserver.cc:460] Webserver started at http://127.1.127.195:45371/ using document root <none> and password file <none>
I20260807 15:05:55.232393 20201 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:55.232486 20201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:55.232651 20201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:55.234303 20201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "1129f351d5274be6818ea490b7d90906"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "1ab41d3c48fe7875a04bcd4435a14d8f"
server_key_iv: "7b9018b47ba5f4cdd30b4a3e5ef3c530"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.234791 20201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "1129f351d5274be6818ea490b7d90906"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "1ab41d3c48fe7875a04bcd4435a14d8f"
server_key_iv: "7b9018b47ba5f4cdd30b4a3e5ef3c530"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.236606 20201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:05:55.237663 20216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.237972 20201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:55.238116 20201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "1129f351d5274be6818ea490b7d90906"
format_stamp: "Formatted at 2026-08-07 15:05:55 on dist-test-slave-8tdl"
server_key: "1ab41d3c48fe7875a04bcd4435a14d8f"
server_key_iv: "7b9018b47ba5f4cdd30b4a3e5ef3c530"
server_key_version: "encryptionkey@0"
I20260807 15:05:55.238216 20201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:55.259254 20201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:55.259712 20201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:55.260124 20201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:55.260496 20201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:55.261025 20201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:55.261099 20201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.261140 20201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:55.261188 20201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:55.269315 20201 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:41647
I20260807 15:05:55.269362 20331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:41647 every 8 connection(s)
I20260807 15:05:55.269791 20201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:55.275745 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20201
I20260807 15:05:55.275876 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:05:55.276121 1535 external_mini_cluster.cc:1428] Setting key 309e371662d4525f8a61e76e1f8b67a5
I20260807 15:05:55.276364 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.270734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:57947 (local address 127.1.127.254:46391)
0807 15:05:55.270786 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.270789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.271078 (+ 289us) server_negotiation.cc:407] Connection header received
0807 15:05:55.271790 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.271791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.271806 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.271872 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:55.272756 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.273303 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.274413 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.274660 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.274778 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.275960 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:55.275962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:55.275962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:55.275969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:55.275988 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:55.275994 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:55.276057 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.276200 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.276242 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:05:55.276633 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.270609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46391 (local address 127.1.127.195:57947)
0807 15:05:55.270935 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.270975 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.271661 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.272049 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.272054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.272063 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:55.272643 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.272648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.273429 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.273434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.274303 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.274305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.274794 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.274795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.274967 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.275748 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:55.275749 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:55.275781 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:55.275810 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:55.276079 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:55.276090 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:55.276094 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.276196 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.276347 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":139,"threads_started":1}
I20260807 15:05:55.277714 20332 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46391
I20260807 15:05:55.278014 20332 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:55.278290 20332 heartbeater.cc:511] Master 127.1.127.254:46391 requested a full tablet report, sending...
I20260807 15:05:55.278838 19867 ts_manager.cc:194] Registered new tserver with Master: 1129f351d5274be6818ea490b7d90906 (127.1.127.195:41647)
I20260807 15:05:55.279276 19867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.195:57947
I20260807 15:05:55.288636 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:05:55.293613 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.290414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.1.127.254:46391)
0807 15:05:55.290512 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.290514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.290528 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:05:55.290642 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.290644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.290660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.290802 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:55.291048 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.291558 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.292353 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.292556 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.292619 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.293357 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:55.293358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:55.293358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:55.293363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:55.293376 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:55.293380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:55.293424 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.293512 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.293540 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:05:55.295799 19867 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:05:55.297714 19867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:05:55.305466 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41647 (local address 127.0.0.1:48908)
0807 15:05:55.300607 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.300658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.300735 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.301203 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.301215 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.301217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:55.301327 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.301484 (+ 157us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.302973 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.302974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.304228 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.304230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.305078 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.305079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.305204 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.305226 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.305286 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.305350 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260807 15:05:55.305696 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45659 (local address 127.0.0.1:56898)
0807 15:05:55.300790 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.300800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.300854 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.301201 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.301204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.301212 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:55.301346 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.301349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.302439 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.302440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.304132 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.304137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.305385 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.305386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.305442 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.305447 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.305530 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.305592 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":73,"thread_start_us":43,"threads_started":1}
I20260807 15:05:55.306176 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.1.127.195:41647)
0807 15:05:55.300998 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.301003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.301015 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:05:55.301068 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.301070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.301087 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.301601 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:55.301677 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.302859 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.304306 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.304956 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.305766 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.305986 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.306020 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.306074 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":102,"threads_started":1}
I20260807 15:05:55.306334 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40313 (local address 127.0.0.1:40446)
0807 15:05:55.300719 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.300732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.301125 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.301638 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.301639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.301642 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:55.301683 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.301685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.303525 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.303526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.304797 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.304801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.306070 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.306071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.306143 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.306147 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.306203 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.306255 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":2,"thread_start_us":128,"threads_started":1}
I20260807 15:05:55.306193 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.1.127.194:45659)
0807 15:05:55.301028 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.301032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.301040 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:05:55.301090 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.301092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.301107 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.301192 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:55.301452 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.302331 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.304264 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.305113 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.305227 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.305348 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.306010 (+ 662us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.306082 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":163,"threads_started":1}
I20260807 15:05:55.306761 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.300504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40446 (local address 127.1.127.193:40313)
0807 15:05:55.300742 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.300748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.301070 (+ 322us) server_negotiation.cc:407] Connection header received
0807 15:05:55.301464 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.301467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.301484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.301562 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:55.301790 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.303407 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.304920 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.305870 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.305944 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.306030 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.306570 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.306633 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
I20260807 15:05:55.307328 20264 tablet_service.cc:1467] Processing CreateTablet for tablet baa042c1097f4f7993283625bfcbe825 (DEFAULT_TABLE table=test-table [id=f88bcabbe478410f8d6c8e9d466caf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:55.307343 20131 tablet_service.cc:1467] Processing CreateTablet for tablet baa042c1097f4f7993283625bfcbe825 (DEFAULT_TABLE table=test-table [id=f88bcabbe478410f8d6c8e9d466caf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:55.307579 19995 tablet_service.cc:1467] Processing CreateTablet for tablet baa042c1097f4f7993283625bfcbe825 (DEFAULT_TABLE table=test-table [id=f88bcabbe478410f8d6c8e9d466caf9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:55.307963 20264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa042c1097f4f7993283625bfcbe825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:55.307978 19995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa042c1097f4f7993283625bfcbe825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:55.308018 20131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baa042c1097f4f7993283625bfcbe825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:55.311668 20346 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Bootstrap starting.
I20260807 15:05:55.312366 20348 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Bootstrap starting.
I20260807 15:05:55.312597 20346 tablet_bootstrap.cc:654] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:55.312978 20346 log.cc:824] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:55.313149 20348 tablet_bootstrap.cc:654] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:55.313503 20348 log.cc:824] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:55.313891 20347 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: Bootstrap starting.
I20260807 15:05:55.314098 20346 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: No bootstrap required, opened a new log
I20260807 15:05:55.314160 20346 ts_tablet_manager.cc:1397] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:05:55.314522 20347 tablet_bootstrap.cc:654] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:55.314917 20347 log.cc:824] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:55.315096 20348 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: No bootstrap required, opened a new log
I20260807 15:05:55.315176 20348 ts_tablet_manager.cc:1397] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:05:55.316006 20347 tablet_bootstrap.cc:492] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: No bootstrap required, opened a new log
I20260807 15:05:55.316082 20347 ts_tablet_manager.cc:1397] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:05:55.316869 20346 raft_consensus.cc:348] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.317077 20346 raft_consensus.cc:374] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:55.317119 20346 raft_consensus.cc:729] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1129f351d5274be6818ea490b7d90906, State: Initialized, Role: FOLLOWER
I20260807 15:05:55.317276 20346 consensus_queue.cc:260] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.317643 20346 ts_tablet_manager.cc:1428] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 15:05:55.317786 20332 heartbeater.cc:503] Master 127.1.127.254:46391 was elected leader, sending a full tablet report...
I20260807 15:05:55.318265 20348 raft_consensus.cc:348] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.318455 20348 raft_consensus.cc:374] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:55.318507 20348 raft_consensus.cc:729] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e8dc734e5848888289d0bdb43530ae, State: Initialized, Role: FOLLOWER
I20260807 15:05:55.318625 20348 consensus_queue.cc:260] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.318957 20348 ts_tablet_manager.cc:1428] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:05:55.319015 20198 heartbeater.cc:503] Master 127.1.127.254:46391 was elected leader, sending a full tablet report...
I20260807 15:05:55.319111 20347 raft_consensus.cc:348] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.319274 20347 raft_consensus.cc:374] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:55.319315 20347 raft_consensus.cc:729] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c7dcf9975941869f11c1cffab5359d, State: Initialized, Role: FOLLOWER
I20260807 15:05:55.319444 20347 consensus_queue.cc:260] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.319749 20347 ts_tablet_manager.cc:1428] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260807 15:05:55.319808 20063 heartbeater.cc:503] Master 127.1.127.254:46391 was elected leader, sending a full tablet report...
I20260807 15:05:55.333878 20354 raft_consensus.cc:482] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:55.334029 20354 raft_consensus.cc:504] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.334376 20354 leader_election.cc:288] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1129f351d5274be6818ea490b7d90906 (127.1.127.195:41647), 32e8dc734e5848888289d0bdb43530ae (127.1.127.194:45659)
I20260807 15:05:55.337662 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.334607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41647 (local address 127.1.127.193:42703)
0807 15:05:55.334700 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.334714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.334782 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.334997 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.334998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.335000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:55.335078 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.335080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.335799 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.335800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.336664 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.336668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.337288 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.337288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.337365 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.337368 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.337447 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.337569 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":24}
I20260807 15:05:55.337661 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.334765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:42703 (local address 127.1.127.195:41647)
0807 15:05:55.334872 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.334874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.334882 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:05:55.334916 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.334918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.334928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.334990 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:55.335174 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.335686 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.336788 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.337191 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.337288 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.337297 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.337454 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.337574 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:05:55.338171 20284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa042c1097f4f7993283625bfcbe825" candidate_uuid: "b0c7dcf9975941869f11c1cffab5359d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1129f351d5274be6818ea490b7d90906" is_pre_election: true
I20260807 15:05:55.338256 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.334770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:45659 (local address 127.1.127.193:37341)
0807 15:05:55.334974 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.334992 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:55.335078 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:55.335209 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:55.335210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:55.335211 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:55.335291 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.335293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.335956 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.335957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.337037 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:55.337042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.337922 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.337926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:55.338032 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.338036 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:55.338107 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:05:55.338154 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":152,"threads_started":1}
I20260807 15:05:55.338294 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.334877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:37341 (local address 127.1.127.194:45659)
0807 15:05:55.334934 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.334936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.335063 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:05:55.335136 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.335137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.335146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.335205 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:55.335368 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.335856 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.337174 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.337630 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.337733 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.337743 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.338160 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.338216 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:05:55.338389 20284 raft_consensus.cc:2454] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c7dcf9975941869f11c1cffab5359d in term 0.
I20260807 15:05:55.338639 20151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa042c1097f4f7993283625bfcbe825" candidate_uuid: "b0c7dcf9975941869f11c1cffab5359d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e8dc734e5848888289d0bdb43530ae" is_pre_election: true
I20260807 15:05:55.338754 19948 leader_election.cc:302] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1129f351d5274be6818ea490b7d90906, b0c7dcf9975941869f11c1cffab5359d; no voters:
I20260807 15:05:55.338830 20151 raft_consensus.cc:2454] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0c7dcf9975941869f11c1cffab5359d in term 0.
I20260807 15:05:55.338930 20354 raft_consensus.cc:2781] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:05:55.339004 20354 raft_consensus.cc:482] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:05:55.339031 20354 raft_consensus.cc:3037] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:55.340050 20354 raft_consensus.cc:504] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.340235 20354 leader_election.cc:288] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [CANDIDATE]: Term 1 election: Requested vote from peers 1129f351d5274be6818ea490b7d90906 (127.1.127.195:41647), 32e8dc734e5848888289d0bdb43530ae (127.1.127.194:45659)
I20260807 15:05:55.340497 20284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa042c1097f4f7993283625bfcbe825" candidate_uuid: "b0c7dcf9975941869f11c1cffab5359d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1129f351d5274be6818ea490b7d90906"
I20260807 15:05:55.340547 20151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baa042c1097f4f7993283625bfcbe825" candidate_uuid: "b0c7dcf9975941869f11c1cffab5359d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e8dc734e5848888289d0bdb43530ae"
I20260807 15:05:55.340607 20284 raft_consensus.cc:3037] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:55.340648 20151 raft_consensus.cc:3037] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:55.341614 20151 raft_consensus.cc:2454] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c7dcf9975941869f11c1cffab5359d in term 1.
I20260807 15:05:55.341650 20284 raft_consensus.cc:2454] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0c7dcf9975941869f11c1cffab5359d in term 1.
I20260807 15:05:55.341822 19948 leader_election.cc:302] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e8dc734e5848888289d0bdb43530ae, b0c7dcf9975941869f11c1cffab5359d; no voters:
I20260807 15:05:55.341992 20354 raft_consensus.cc:2781] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:55.342240 20354 raft_consensus.cc:686] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [term 1 LEADER]: Becoming Leader. State: Replica: b0c7dcf9975941869f11c1cffab5359d, State: Running, Role: LEADER
I20260807 15:05:55.342362 20354 consensus_queue.cc:237] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } }
I20260807 15:05:55.343667 19867 catalog_manager.cc:5495] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d reported cstate change: term changed from 0 to 1, leader changed from <none> to b0c7dcf9975941869f11c1cffab5359d (127.1.127.193). New cstate: current_term: 1 leader_uuid: "b0c7dcf9975941869f11c1cffab5359d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0c7dcf9975941869f11c1cffab5359d" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40313 } health_report { overall_health: HEALTHY } } }
I20260807 15:05:55.354242 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.351586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.1.127.193:40313)
0807 15:05:55.351667 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:55.351669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:55.351679 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:55.351733 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:55.351734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:55.351756 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:55.351838 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:55.351962 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.352527 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.353342 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:55.353553 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:55.353633 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:55.353833 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:55.353946 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:05:55.354064 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:55.354085 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 15:05:55.354109 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:05:55.356479 20151 raft_consensus.cc:1261] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 1 FOLLOWER]: Refusing update from remote peer b0c7dcf9975941869f11c1cffab5359d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:55.356519 20284 raft_consensus.cc:1261] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 1 FOLLOWER]: Refusing update from remote peer b0c7dcf9975941869f11c1cffab5359d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:55.356957 20356 consensus_queue.cc:1035] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32e8dc734e5848888289d0bdb43530ae" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 45659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:55.357244 20354 consensus_queue.cc:1035] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1129f351d5274be6818ea490b7d90906" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:55.360160 20361 mvcc.cc:205] Tried to move back new op lower bound from 7315927676336668672 to 7315927676283203584. Current Snapshot: MvccSnapshot[applied={T|T < 7315927676336668672}]
I20260807 15:05:55.360848 20360 mvcc.cc:205] Tried to move back new op lower bound from 7315927676336668672 to 7315927676283203584. Current Snapshot: MvccSnapshot[applied={T|T < 7315927676336668672 or (T in {7315927676336668672})}]
I20260807 15:05:55.362938 19867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:55.363054 19867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:55.364035 19867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4ef5a86f93347bcbcf44047f057d8cd: Sending DeleteTablet for 3 replicas of tablet baa042c1097f4f7993283625bfcbe825
I20260807 15:05:55.364449 20264 tablet_service.cc:1514] Processing DeleteTablet for tablet baa042c1097f4f7993283625bfcbe825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:55 UTC) from {username='slave'} at 127.0.0.1:48908
I20260807 15:05:55.364612 20131 tablet_service.cc:1514] Processing DeleteTablet for tablet baa042c1097f4f7993283625bfcbe825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:55 UTC) from {username='slave'} at 127.0.0.1:56898
I20260807 15:05:55.364579 19995 tablet_service.cc:1514] Processing DeleteTablet for tablet baa042c1097f4f7993283625bfcbe825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:55 UTC) from {username='slave'} at 127.0.0.1:40446
I20260807 15:05:55.364863 20372 tablet_replica.cc:330] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: stopping tablet replica
I20260807 15:05:55.365037 20372 raft_consensus.cc:2229] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:55.365175 20372 raft_consensus.cc:2258] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:55.365366 20371 tablet_replica.cc:330] T baa042c1097f4f7993283625bfcbe825 P b0c7dcf9975941869f11c1cffab5359d: stopping tablet replica
I20260807 15:05:55.365461 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19930
I20260807 15:05:55.366257 20370 tablet_replica.cc:330] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: stopping tablet replica
I20260807 15:05:55.366411 20370 raft_consensus.cc:2229] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:55.366564 20370 raft_consensus.cc:2258] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:55.367810 20370 ts_tablet_manager.cc:1905] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:55.368564 20372 ts_tablet_manager.cc:1905] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:55.370332 20370 ts_tablet_manager.cc:1918] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:55.370437 20370 log.cc:1196] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/baa042c1097f4f7993283625bfcbe825
I20260807 15:05:55.370810 20370 ts_tablet_manager.cc:1939] T baa042c1097f4f7993283625bfcbe825 P 1129f351d5274be6818ea490b7d90906: Deleting consensus metadata
I20260807 15:05:55.371337 20372 ts_tablet_manager.cc:1918] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:55.371430 20372 log.cc:1196] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/baa042c1097f4f7993283625bfcbe825
I20260807 15:05:55.371412 19852 catalog_manager.cc:4841] TS 1129f351d5274be6818ea490b7d90906 (127.1.127.195:41647): tablet baa042c1097f4f7993283625bfcbe825 (table test-table [id=f88bcabbe478410f8d6c8e9d466caf9c]) successfully deleted
I20260807 15:05:55.371730 20372 ts_tablet_manager.cc:1939] T baa042c1097f4f7993283625bfcbe825 P 32e8dc734e5848888289d0bdb43530ae: Deleting consensus metadata
I20260807 15:05:55.372217 19852 catalog_manager.cc:4841] TS 32e8dc734e5848888289d0bdb43530ae (127.1.127.194:45659): tablet baa042c1097f4f7993283625bfcbe825 (table test-table [id=f88bcabbe478410f8d6c8e9d466caf9c]) successfully deleted
W20260807 15:05:55.375344 19851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:40313 (error 108)
I20260807 15:05:55.375614 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20067
I20260807 15:05:55.377426 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:55.377208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40313 (local address 127.0.0.1:40476)
0807 15:05:55.377292 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:55.377372 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:40313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:05:55.378394 19851 catalog_manager.cc:4584] TS b0c7dcf9975941869f11c1cffab5359d (127.1.127.193:40313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet baa042c1097f4f7993283625bfcbe825: Network error: Client connection negotiation failed: client connection to 127.1.127.193:40313: connect: Connection refused (error 111)
I20260807 15:05:55.382243 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20201
I20260807 15:05:55.388278 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 19835
2026-08-07T15:05:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (783 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:05:55 dist-test-slave-8tdl krb5kdc[20376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:05:55 dist-test-slave-8tdl krb5kdc[20376](info): set up 2 sockets
Aug 07 15:05:55 dist-test-slave-8tdl krb5kdc[20376](info): commencing operation
krb5kdc: starting...
W20260807 15:05:56.489573 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:05:56 dist-test-slave-8tdl krb5kdc[20376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:05:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:05:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:05:56.638375 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39921
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:39921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:56.724877 20390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:56.725152 20390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:56.725195 20390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:56.726462 20390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:56.726552 20390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:56.726577 20390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:56.726599 20390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:56.726630 20390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:56.728909 20390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:39921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:56.729230 20390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:56.729496 20390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:56.734870 20398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:56.734828 20395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:56.734838 20396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:56.735049 20390 server_base.cc:1034] running on GCE node
I20260807 15:05:56.735352 20390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:56.735666 20390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:56.736833 20390 hybrid_clock.cc:648] HybridClock initialized: now 1786115156736806 us; error 37 us; skew 500 ppm
Aug 07 15:05:56 dist-test-slave-8tdl krb5kdc[20376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:56.746513 20390 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:05:56.747265 20390 webserver.cc:460] Webserver started at http://127.1.127.254:36587/ using document root <none> and password file <none>
I20260807 15:05:56.747608 20390 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:56.747668 20390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:56.747830 20390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:56.749154 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "844d8ded8ebb4eeca8be91432fa9ff63"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "31566eec7372901c1c4933fea8dabcb4"
server_key_iv: "814743520617027c1faa070befea7978"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.749631 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "844d8ded8ebb4eeca8be91432fa9ff63"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "31566eec7372901c1c4933fea8dabcb4"
server_key_iv: "814743520617027c1faa070befea7978"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.751365 20390 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:05:56.752517 20405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:56.752849 20390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:56.752936 20390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "844d8ded8ebb4eeca8be91432fa9ff63"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "31566eec7372901c1c4933fea8dabcb4"
server_key_iv: "814743520617027c1faa070befea7978"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.753053 20390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:56.767799 20390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:56.770139 20390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:56.770308 20390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:56.775245 20390 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:39921
I20260807 15:05:56.775326 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:39921 every 8 connection(s)
I20260807 15:05:56.775743 20390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:56.776763 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:56.779548 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: Bootstrap starting.
I20260807 15:05:56.780264 20459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:56.780609 20459 log.cc:824] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:56.781538 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: No bootstrap required, opened a new log
I20260807 15:05:56.783535 20459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } }
I20260807 15:05:56.783702 20459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:56.783726 20459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844d8ded8ebb4eeca8be91432fa9ff63, State: Initialized, Role: FOLLOWER
I20260807 15:05:56.783905 20459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } }
I20260807 15:05:56.783994 20459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:56.784041 20459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:56.784093 20459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:56.784809 20459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } }
I20260807 15:05:56.784984 20459 leader_election.cc:302] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 844d8ded8ebb4eeca8be91432fa9ff63; no voters:
I20260807 15:05:56.785104 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20390
I20260807 15:05:56.785214 20459 leader_election.cc:288] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:05:56.785224 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:05:56.785291 20462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:56.785508 1535 external_mini_cluster.cc:1428] Setting key 1b7c44c65958ba36366319d482f0969e
I20260807 15:05:56.785539 20462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [term 1 LEADER]: Becoming Leader. State: Replica: 844d8ded8ebb4eeca8be91432fa9ff63, State: Running, Role: LEADER
I20260807 15:05:56.785580 20459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:56.785645 20462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } }
I20260807 15:05:56.785982 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 844d8ded8ebb4eeca8be91432fa9ff63. Latest consensus state: current_term: 1 leader_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } } }
I20260807 15:05:56.786072 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:56.786387 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844d8ded8ebb4eeca8be91432fa9ff63" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39921 } } }
I20260807 15:05:56.786541 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:56.787343 20468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:56.787921 20468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:56.789680 20468 catalog_manager.cc:1347] Generated new cluster ID: ad6ce5c195b949a68d7661e3a67b03b6
I20260807 15:05:56.789746 20468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:05:56 dist-test-slave-8tdl krb5kdc[20376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:56.797151 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:56.786416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.1.127.254:39921)
0807 15:05:56.786929 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:56.786940 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:56.786979 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:05:56.787118 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:56.787131 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:56.787205 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:56.787408 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:56.788295 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:56.789585 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:56.790454 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:56.790998 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:56.791123 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:56.793641 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:56.793645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:56.793646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:56.793659 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:56.795481 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:56.795879 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:56.795883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:56.795883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:56.796000 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:56.796232 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:56.796233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:56.796233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:56.796514 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:56.796705 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:56.796775 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:05:56.796926 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":112,"threads_started":1}
I20260807 15:05:56.814440 20468 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:05:56.815119 20468 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:56.821483 20468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: Generated new TSK 0
I20260807 15:05:56.821753 20468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:05:56.859766 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:56.950353 20485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:56.950621 20485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:56.950651 20485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:56.952029 20485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:56.952091 20485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:56.952123 20485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:56.953773 20485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:56.954063 20485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:56.954358 20485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:56.960318 20490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:56.960345 20491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:56.960318 20493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:56.960377 20485 server_base.cc:1034] running on GCE node
I20260807 15:05:56.960893 20485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:56.961185 20485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:56.962369 20485 hybrid_clock.cc:648] HybridClock initialized: now 1786115156962350 us; error 39 us; skew 500 ppm
Aug 07 15:05:56 dist-test-slave-8tdl krb5kdc[20376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:56.965329 20485 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:05:56.965854 20485 webserver.cc:460] Webserver started at http://127.1.127.193:40005/ using document root <none> and password file <none>
I20260807 15:05:56.966115 20485 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:56.966189 20485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:56.966339 20485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:56.967505 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "b0b627a0d3ca42b3a79088a92b44231f"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "644c0273c677999b7656b3096768d0fe"
server_key_iv: "a1dcbab125e439b96acf3260d7451736"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.967873 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b627a0d3ca42b3a79088a92b44231f"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "644c0273c677999b7656b3096768d0fe"
server_key_iv: "a1dcbab125e439b96acf3260d7451736"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.969316 20485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:05:56.970274 20500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:56.970614 20485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:56.970762 20485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "b0b627a0d3ca42b3a79088a92b44231f"
format_stamp: "Formatted at 2026-08-07 15:05:56 on dist-test-slave-8tdl"
server_key: "644c0273c677999b7656b3096768d0fe"
server_key_iv: "a1dcbab125e439b96acf3260d7451736"
server_key_version: "encryptionkey@0"
I20260807 15:05:56.970832 20485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:56.989217 20485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:56.991650 20485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:56.991786 20485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:56.992053 20485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:56.992453 20485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:56.992504 20485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:56.992554 20485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:56.992581 20485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:56.999362 20485 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:43533
I20260807 15:05:56.999502 20618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:43533 every 8 connection(s)
I20260807 15:05:56.999789 20485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:05:57 dist-test-slave-8tdl krb5kdc[20376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:57.007262 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20485
I20260807 15:05:57.007416 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:05:57.007752 1535 external_mini_cluster.cc:1428] Setting key 4e662859ec5db3b15c7c99234d42fad4
I20260807 15:05:57.011782 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.000568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39921 (local address 127.1.127.193:51931)
0807 15:05:57.000790 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.000830 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.001365 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.001618 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.001622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.002226 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:57.002438 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.002442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.003315 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.003316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.004180 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.004183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.004649 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.004651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.004787 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.005378 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:57.005387 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:57.007082 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:57.009476 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.009502 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.009506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.009790 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.010166 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.010168 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.010168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.010313 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.010825 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:57.010837 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:57.011097 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.011220 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.011470 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":128,"threads_started":1}
I20260807 15:05:57.012372 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.000597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:51931 (local address 127.1.127.254:39921)
0807 15:05:57.000679 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.000682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.000942 (+ 260us) server_negotiation.cc:407] Connection header received
0807 15:05:57.001486 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.001488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.001502 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.001563 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:57.002548 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.003205 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.004280 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.004494 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.004559 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.007213 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:57.007214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:57.007215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:57.007224 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:57.009319 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.009992 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.009993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.009994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.010077 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.010404 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.010405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.010405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.010645 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:57.011994 (+ 1349us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.012141 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.012282 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:05:57.012775 20619 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39921
I20260807 15:05:57.012882 20619 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:57.013149 20619 heartbeater.cc:511] Master 127.1.127.254:39921 requested a full tablet report, sending...
I20260807 15:05:57.013752 20422 ts_manager.cc:194] Registered new tserver with Master: b0b627a0d3ca42b3a79088a92b44231f (127.1.127.193:43533)
I20260807 15:05:57.014426 20422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:51931
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:05:57.043030 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:57.131228 20624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:57.131460 20624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:57.131486 20624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:57.132647 20624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:57.132691 20624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:57.132719 20624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:57.134501 20624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:57.134876 20624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:57.135102 20624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:57.141038 20630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.141063 20632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.141028 20629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:57.141403 20624 server_base.cc:1034] running on GCE node
I20260807 15:05:57.141649 20624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:57.142081 20624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:57.143262 20624 hybrid_clock.cc:648] HybridClock initialized: now 1786115157143231 us; error 51 us; skew 500 ppm
Aug 07 15:05:57 dist-test-slave-8tdl krb5kdc[20376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115157, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:57.146431 20624 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:05:57.147037 20624 webserver.cc:460] Webserver started at http://127.1.127.194:38951/ using document root <none> and password file <none>
I20260807 15:05:57.147311 20624 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:57.147362 20624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:57.147521 20624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:57.148710 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "cb355b5444d34464a70218536a3d500e"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "cd3a7cbd4c04357f4f3c51070a2dd4f4"
server_key_iv: "5f6cd868852bb00dd7bb5d03881f6096"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.149103 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "cb355b5444d34464a70218536a3d500e"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "cd3a7cbd4c04357f4f3c51070a2dd4f4"
server_key_iv: "5f6cd868852bb00dd7bb5d03881f6096"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.150646 20624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:05:57.151679 20639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.151882 20624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:57.151999 20624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "cb355b5444d34464a70218536a3d500e"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "cd3a7cbd4c04357f4f3c51070a2dd4f4"
server_key_iv: "5f6cd868852bb00dd7bb5d03881f6096"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.152151 20624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:57.191736 20624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:57.194166 20624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:57.194371 20624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:57.194691 20624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:57.195058 20624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:57.195117 20624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.195155 20624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:57.195183 20624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.201318 20624 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:37827
I20260807 15:05:57.201387 20756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:37827 every 8 connection(s)
I20260807 15:05:57.201768 20624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:05:57 dist-test-slave-8tdl krb5kdc[20376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115157, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:57.210000 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20624
I20260807 15:05:57.210153 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:05:57.210418 1535 external_mini_cluster.cc:1428] Setting key e7105697662e1f5565167b2d2007fede
I20260807 15:05:57.212464 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.202649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:50143 (local address 127.1.127.254:39921)
0807 15:05:57.202743 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.202745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.202989 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:05:57.203450 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.203452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.203465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.203540 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:57.204531 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.205317 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.206516 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.206808 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.206872 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.209238 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:57.209240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:57.209240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:57.209247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:57.210979 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.211419 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.211420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.211420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.211488 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.211754 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.211755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.211755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.211893 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:57.211967 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.212261 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.212359 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:05:57.212663 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.202618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39921 (local address 127.1.127.194:50143)
0807 15:05:57.202810 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.202856 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.203355 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.203595 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.203598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.204190 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:57.204394 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.204399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.205463 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.205464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.206393 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.206397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.206901 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.206902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.207031 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.207547 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:57.207555 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:57.209126 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:57.211077 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.211085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.211088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.211332 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.211562 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.211563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.211563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.211683 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.211969 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:57.211977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:57.212119 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.212228 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.212407 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260807 15:05:57.213433 20757 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39921
I20260807 15:05:57.213550 20757 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:57.213783 20757 heartbeater.cc:511] Master 127.1.127.254:39921 requested a full tablet report, sending...
I20260807 15:05:57.214299 20422 ts_manager.cc:194] Registered new tserver with Master: cb355b5444d34464a70218536a3d500e (127.1.127.194:37827)
I20260807 15:05:57.214833 20422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:50143
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:05:57.243594 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:57.338902 20762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:57.339251 20762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:57.339303 20762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:57.341146 20762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:57.341254 20762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:57.341312 20762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:57.343246 20762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:57.343621 20762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:57.343928 20762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:57.349893 20769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.349882 20771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.349895 20767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:57.350245 20762 server_base.cc:1034] running on GCE node
I20260807 15:05:57.350489 20762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:57.350840 20762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:57.352128 20762 hybrid_clock.cc:648] HybridClock initialized: now 1786115157352112 us; error 36 us; skew 500 ppm
Aug 07 15:05:57 dist-test-slave-8tdl krb5kdc[20376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115157, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:57.355224 20762 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:05:57.355878 20762 webserver.cc:460] Webserver started at http://127.1.127.195:35223/ using document root <none> and password file <none>
I20260807 15:05:57.356168 20762 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:57.356236 20762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:57.356380 20762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:57.357632 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "2c7797f7021d40c6b02dd0dabc322909"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "1b44edfddb6066f52dba80f52a693e8f"
server_key_iv: "c7bedfd284365a2c484bb89e1034c555"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.358026 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "2c7797f7021d40c6b02dd0dabc322909"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "1b44edfddb6066f52dba80f52a693e8f"
server_key_iv: "c7bedfd284365a2c484bb89e1034c555"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.359571 20762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:05:57.360509 20778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.360737 20762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:57.360853 20762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "2c7797f7021d40c6b02dd0dabc322909"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "1b44edfddb6066f52dba80f52a693e8f"
server_key_iv: "c7bedfd284365a2c484bb89e1034c555"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.360940 20762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:57.379287 20762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:57.381888 20762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:57.382081 20762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:57.382340 20762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:57.382725 20762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:57.382788 20762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.382831 20762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:57.382861 20762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.389281 20762 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:38905
I20260807 15:05:57.389333 20895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:38905 every 8 connection(s)
I20260807 15:05:57.389678 20762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:57.390204 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20762
I20260807 15:05:57.390300 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:05:57.390553 1535 external_mini_cluster.cc:1428] Setting key 316ec7d7f14a4cdf0790aadf004314a5
Aug 07 15:05:57 dist-test-slave-8tdl krb5kdc[20376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115157, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:57.400068 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.390565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:60929 (local address 127.1.127.254:39921)
0807 15:05:57.390647 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.390650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.390889 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:05:57.391633 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.391634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.391647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.391710 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:57.392767 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.393335 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.394357 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.394568 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.394625 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.397074 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:57.397075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:57.397075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:57.397082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:57.398663 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.399131 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.399132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.399133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.399189 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.399445 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.399446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.399447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.399576 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:57.399644 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.399932 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.400002 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:05:57.400156 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.390516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39921 (local address 127.1.127.195:60929)
0807 15:05:57.390703 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.390750 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.391519 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.391796 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.391805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.392453 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:57.392669 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.392675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.393467 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.393468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.394273 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.394276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.394645 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.394646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.394875 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.395475 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:05:57.395484 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:57.396947 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:57.398806 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.398813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.398815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.399060 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.399251 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:05:57.399252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:05:57.399252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:05:57.399356 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:05:57.399645 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:57.399649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:57.399770 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.399853 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.399984 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260807 15:05:57.400683 20896 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39921
I20260807 15:05:57.400796 20896 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:57.401047 20896 heartbeater.cc:511] Master 127.1.127.254:39921 requested a full tablet report, sending...
I20260807 15:05:57.401566 20422 ts_manager.cc:194] Registered new tserver with Master: 2c7797f7021d40c6b02dd0dabc322909 (127.1.127.195:38905)
I20260807 15:05:57.402064 20422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:60929
I20260807 15:05:57.402899 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:05:57.411588 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.404351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.1.127.254:39921)
0807 15:05:57.404403 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.404405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.404497 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:05:57.404545 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.404545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.404554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.404604 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:57.405221 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.405728 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.406586 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.406857 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.406973 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.408535 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:57.408537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:57.408537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:57.408544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:57.410098 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.410542 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.410543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.410543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.410612 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:57.410891 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:57.410892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:57.410892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:57.411044 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:57.411207 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.411404 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.411464 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:05:57.413704 20421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:05:57.415616 20421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:05:57.423663 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.418864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.1.127.195:38905)
0807 15:05:57.419142 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.419146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.419155 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:05:57.419194 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.419195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.419210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.419562 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:57.419648 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.420759 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.422304 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.423158 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.423243 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.423310 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.423507 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.423557 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":178,"threads_started":1}
I20260807 15:05:57.423655 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.418578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38905 (local address 127.0.0.1:44380)
0807 15:05:57.418761 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.418771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.418812 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.419313 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.419314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.419316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:57.419445 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.419447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.421153 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.421154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.422195 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.422198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.423275 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.423277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.423369 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.423390 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.423468 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.423521 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":51,"thread_start_us":528,"threads_started":1}
I20260807 15:05:57.423977 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.418824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.1.127.193:43533)
0807 15:05:57.419001 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.419006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.419014 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:05:57.419072 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.419074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.419094 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.419216 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:57.419475 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.420759 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.422762 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.423545 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.423625 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.423682 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.423850 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.423899 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260807 15:05:57.423966 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.418466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:43533 (local address 127.0.0.1:36852)
0807 15:05:57.418595 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.418639 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.418738 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.419263 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.419267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.419271 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:57.419375 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.419378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.420881 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.420882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.422339 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.422342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.423668 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.423670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.423744 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.423749 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.423829 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.423888 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":86,"thread_start_us":78,"threads_started":1}
I20260807 15:05:57.424772 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.418578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37827 (local address 127.0.0.1:44324)
0807 15:05:57.419331 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.419345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.419401 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.419917 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.419918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.419921 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:57.419967 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.419968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.420992 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.420992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.422339 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.422342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.424502 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.424503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.424582 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.424587 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.424644 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.424696 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":43,"thread_start_us":133,"threads_started":1}
I20260807 15:05:57.424793 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.419133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.1.127.194:37827)
0807 15:05:57.419642 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.419645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.419656 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:05:57.419689 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.419691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.419704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.419829 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:57.420144 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.420894 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.422437 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.424347 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.424415 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.424468 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.424665 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.424710 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":93,"threads_started":1}
I20260807 15:05:57.425160 20550 tablet_service.cc:1467] Processing CreateTablet for tablet faf221456a414cef88d3925c58270066 (DEFAULT_TABLE table=test-table [id=7fe53cea32934fbaae74e908414cdb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:57.425153 20829 tablet_service.cc:1467] Processing CreateTablet for tablet faf221456a414cef88d3925c58270066 (DEFAULT_TABLE table=test-table [id=7fe53cea32934fbaae74e908414cdb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:57.425417 20691 tablet_service.cc:1467] Processing CreateTablet for tablet faf221456a414cef88d3925c58270066 (DEFAULT_TABLE table=test-table [id=7fe53cea32934fbaae74e908414cdb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:57.425557 20550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf221456a414cef88d3925c58270066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:57.425558 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf221456a414cef88d3925c58270066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:57.425670 20691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf221456a414cef88d3925c58270066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:57.428742 20914 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Bootstrap starting.
I20260807 15:05:57.429435 20914 tablet_bootstrap.cc:654] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:57.429397 20915 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Bootstrap starting.
I20260807 15:05:57.430079 20914 log.cc:824] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:57.430137 20915 tablet_bootstrap.cc:654] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:57.430555 20916 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Bootstrap starting.
I20260807 15:05:57.430733 20915 log.cc:824] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:57.431142 20914 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: No bootstrap required, opened a new log
I20260807 15:05:57.431233 20914 ts_tablet_manager.cc:1397] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:05:57.431471 20916 tablet_bootstrap.cc:654] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:57.431922 20916 log.cc:824] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:57.432663 20915 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: No bootstrap required, opened a new log
I20260807 15:05:57.432754 20915 ts_tablet_manager.cc:1397] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:05:57.432789 20916 tablet_bootstrap.cc:492] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: No bootstrap required, opened a new log
I20260807 15:05:57.432876 20916 ts_tablet_manager.cc:1397] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:05:57.433301 20914 raft_consensus.cc:348] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.433521 20914 raft_consensus.cc:374] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:57.433559 20914 raft_consensus.cc:729] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb355b5444d34464a70218536a3d500e, State: Initialized, Role: FOLLOWER
I20260807 15:05:57.433703 20914 consensus_queue.cc:260] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.434051 20914 ts_tablet_manager.cc:1428] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:05:57.434109 20757 heartbeater.cc:503] Master 127.1.127.254:39921 was elected leader, sending a full tablet report...
I20260807 15:05:57.434859 20915 raft_consensus.cc:348] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.435138 20915 raft_consensus.cc:374] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:57.435031 20916 raft_consensus.cc:348] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.435187 20915 raft_consensus.cc:729] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b627a0d3ca42b3a79088a92b44231f, State: Initialized, Role: FOLLOWER
I20260807 15:05:57.435217 20916 raft_consensus.cc:374] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:57.435266 20916 raft_consensus.cc:729] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7797f7021d40c6b02dd0dabc322909, State: Initialized, Role: FOLLOWER
I20260807 15:05:57.435322 20915 consensus_queue.cc:260] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.435448 20916 consensus_queue.cc:260] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.435621 20915 ts_tablet_manager.cc:1428] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 15:05:57.435784 20619 heartbeater.cc:503] Master 127.1.127.254:39921 was elected leader, sending a full tablet report...
I20260807 15:05:57.435819 20896 heartbeater.cc:503] Master 127.1.127.254:39921 was elected leader, sending a full tablet report...
I20260807 15:05:57.435817 20916 ts_tablet_manager.cc:1428] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260807 15:05:57.452495 20758 tablet.cc:2369] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:05:57.500664 20620 tablet.cc:2369] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:57.636650 20922 raft_consensus.cc:482] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:57.636806 20922 raft_consensus.cc:504] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.637199 20922 leader_election.cc:288] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b627a0d3ca42b3a79088a92b44231f (127.1.127.193:43533), 2c7797f7021d40c6b02dd0dabc322909 (127.1.127.195:38905)
W20260807 15:05:57.640236 20897 tablet.cc:2369] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:57.641808 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.637557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:58639 (local address 127.1.127.193:43533)
0807 15:05:57.637608 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.637610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.637618 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:05:57.637651 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.637652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.637662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.637726 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:57.638090 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.638962 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.640694 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.641308 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.641436 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.641450 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.641687 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.641735 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:05:57.641808 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.637566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:43719 (local address 127.1.127.195:38905)
0807 15:05:57.637619 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.637621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.637972 (+ 351us) server_negotiation.cc:407] Connection header received
0807 15:05:57.638045 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.638046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.638061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.638138 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:57.638316 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.639026 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.640694 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.641421 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.641505 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.641512 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.641687 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.641735 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:05:57.641829 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.637443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:43533 (local address 127.1.127.194:58639)
0807 15:05:57.637496 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.637510 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.637563 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.637805 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.637807 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.637810 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:57.637957 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.637959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.639149 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.639150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.640534 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.640538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.641451 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.641452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.641594 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.641597 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.641664 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.641729 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":109}
I20260807 15:05:57.641840 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.637441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:38905 (local address 127.1.127.194:43719)
0807 15:05:57.637846 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.637858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:57.637937 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:57.638167 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:57.638168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:57.638170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:57.638229 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.638231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.639149 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.639150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.640574 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:57.640576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.641513 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.641513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:57.641587 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.641593 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:05:57.641664 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:05:57.641728 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":190,"thread_start_us":134,"threads_started":1}
I20260807 15:05:57.642400 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf221456a414cef88d3925c58270066" candidate_uuid: "cb355b5444d34464a70218536a3d500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b627a0d3ca42b3a79088a92b44231f" is_pre_election: true
I20260807 15:05:57.642483 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf221456a414cef88d3925c58270066" candidate_uuid: "cb355b5444d34464a70218536a3d500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7797f7021d40c6b02dd0dabc322909" is_pre_election: true
I20260807 15:05:57.642601 20571 raft_consensus.cc:2454] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb355b5444d34464a70218536a3d500e in term 0.
I20260807 15:05:57.642599 20849 raft_consensus.cc:2454] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb355b5444d34464a70218536a3d500e in term 0.
I20260807 15:05:57.642917 20641 leader_election.cc:302] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b627a0d3ca42b3a79088a92b44231f, cb355b5444d34464a70218536a3d500e; no voters:
I20260807 15:05:57.643055 20922 raft_consensus.cc:2781] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:05:57.643116 20922 raft_consensus.cc:482] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:05:57.643139 20922 raft_consensus.cc:3037] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:57.643896 20922 raft_consensus.cc:504] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.644084 20922 leader_election.cc:288] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [CANDIDATE]: Term 1 election: Requested vote from peers b0b627a0d3ca42b3a79088a92b44231f (127.1.127.193:43533), 2c7797f7021d40c6b02dd0dabc322909 (127.1.127.195:38905)
I20260807 15:05:57.644223 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf221456a414cef88d3925c58270066" candidate_uuid: "cb355b5444d34464a70218536a3d500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b627a0d3ca42b3a79088a92b44231f"
I20260807 15:05:57.644248 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf221456a414cef88d3925c58270066" candidate_uuid: "cb355b5444d34464a70218536a3d500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7797f7021d40c6b02dd0dabc322909"
I20260807 15:05:57.644322 20849 raft_consensus.cc:3037] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:57.644308 20571 raft_consensus.cc:3037] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:57.645099 20849 raft_consensus.cc:2454] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb355b5444d34464a70218536a3d500e in term 1.
I20260807 15:05:57.645136 20571 raft_consensus.cc:2454] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb355b5444d34464a70218536a3d500e in term 1.
I20260807 15:05:57.645303 20642 leader_election.cc:302] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c7797f7021d40c6b02dd0dabc322909, cb355b5444d34464a70218536a3d500e; no voters:
I20260807 15:05:57.645426 20922 raft_consensus.cc:2781] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:57.645649 20922 raft_consensus.cc:686] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 1 LEADER]: Becoming Leader. State: Replica: cb355b5444d34464a70218536a3d500e, State: Running, Role: LEADER
I20260807 15:05:57.645771 20922 consensus_queue.cc:237] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } }
I20260807 15:05:57.646808 20422 catalog_manager.cc:5495] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e reported cstate change: term changed from 0 to 1, leader changed from <none> to cb355b5444d34464a70218536a3d500e (127.1.127.194). New cstate: current_term: 1 leader_uuid: "cb355b5444d34464a70218536a3d500e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb355b5444d34464a70218536a3d500e" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 37827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 } health_report { overall_health: UNKNOWN } } }
I20260807 15:05:57.653806 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.650379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.1.127.194:37827)
0807 15:05:57.650598 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.650600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.650614 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:05:57.650659 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.650660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.650720 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.650811 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:57.651140 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.652015 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.652840 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.653106 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.653166 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.653379 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:57.653473 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:05:57.653549 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.653666 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.653699 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260807 15:05:57.656147 20571 raft_consensus.cc:1261] T faf221456a414cef88d3925c58270066 P b0b627a0d3ca42b3a79088a92b44231f [term 1 FOLLOWER]: Refusing update from remote peer cb355b5444d34464a70218536a3d500e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:57.656374 20849 raft_consensus.cc:1261] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 1 FOLLOWER]: Refusing update from remote peer cb355b5444d34464a70218536a3d500e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:57.656611 20926 consensus_queue.cc:1035] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b627a0d3ca42b3a79088a92b44231f" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:57.656790 20922 consensus_queue.cc:1035] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7797f7021d40c6b02dd0dabc322909" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:57.659667 20930 mvcc.cc:205] Tried to move back new op lower bound from 7315927685755928576 to 7315927685717667840. Current Snapshot: MvccSnapshot[applied={T|T < 7315927685755928576}]
I20260807 15:05:57.659857 20931 mvcc.cc:205] Tried to move back new op lower bound from 7315927685755928576 to 7315927685717667840. Current Snapshot: MvccSnapshot[applied={T|T < 7315927685755928576}]
I20260807 15:05:57.660466 20929 mvcc.cc:205] Tried to move back new op lower bound from 7315927685755928576 to 7315927685717667840. Current Snapshot: MvccSnapshot[applied={T|T < 7315927685755928576}]
I20260807 15:05:57.662961 20421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:57.663093 20421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:57.663975 20421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 844d8ded8ebb4eeca8be91432fa9ff63: Sending DeleteTablet for 3 replicas of tablet faf221456a414cef88d3925c58270066
I20260807 15:05:57.664413 20691 tablet_service.cc:1514] Processing DeleteTablet for tablet faf221456a414cef88d3925c58270066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:57 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:44324
I20260807 15:05:57.664469 20550 tablet_service.cc:1514] Processing DeleteTablet for tablet faf221456a414cef88d3925c58270066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:57 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:36852
I20260807 15:05:57.664713 20940 tablet_replica.cc:330] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: stopping tablet replica
I20260807 15:05:57.664779 20829 tablet_service.cc:1514] Processing DeleteTablet for tablet faf221456a414cef88d3925c58270066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:57 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:44380
I20260807 15:05:57.664839 20940 raft_consensus.cc:2229] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:05:57.664978 20940 raft_consensus.cc:2258] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:57.665156 20942 tablet_replica.cc:330] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: stopping tablet replica
I20260807 15:05:57.665349 20942 raft_consensus.cc:2229] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:57.665361 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20485
I20260807 15:05:57.666375 20940 ts_tablet_manager.cc:1905] T faf221456a414cef88d3925c58270066 P cb355b5444d34464a70218536a3d500e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:57.668560 20942 raft_consensus.cc:2258] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:57.669209 20942 ts_tablet_manager.cc:1905] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:57.671175 20942 ts_tablet_manager.cc:1918] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:57.671286 20942 log.cc:1196] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/faf221456a414cef88d3925c58270066
I20260807 15:05:57.671579 20942 ts_tablet_manager.cc:1939] T faf221456a414cef88d3925c58270066 P 2c7797f7021d40c6b02dd0dabc322909: Deleting consensus metadata
I20260807 15:05:57.672055 20408 catalog_manager.cc:4841] TS 2c7797f7021d40c6b02dd0dabc322909 (127.1.127.195:38905): tablet faf221456a414cef88d3925c58270066 (table test-table [id=7fe53cea32934fbaae74e908414cdb82]) successfully deleted
W20260807 15:05:57.676889 20407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:43533 (error 108)
I20260807 15:05:57.677222 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20624
I20260807 15:05:57.683421 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.683236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:43533 (local address 127.0.0.1:36864)
0807 15:05:57.683323 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.683381 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:43533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 15:05:57.683652 20407 catalog_manager.cc:4584] TS b0b627a0d3ca42b3a79088a92b44231f (127.1.127.193:43533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet faf221456a414cef88d3925c58270066: Network error: Client connection negotiation failed: client connection to 127.1.127.193:43533: connect: Connection refused (error 111)
I20260807 15:05:57.684118 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.683929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:37827 (local address 127.0.0.1:44328)
0807 15:05:57.684051 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:57.684083 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.194:37827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 15:05:57.684187 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20762
I20260807 15:05:57.690927 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20390
2026-08-07T15:05:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2302 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T15:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:05:57Z Disabled control of system clock
I20260807 15:05:57.723603 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39577
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:39577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:57.810945 20948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:57.811193 20948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:57.811216 20948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:57.812491 20948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:57.812544 20948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:57.812558 20948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:57.812570 20948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:57.812583 20948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:57.814067 20948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:39577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:39577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:57.814391 20948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:57.814774 20948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:57.820286 20956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.820286 20954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:57.820631 20953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:57.820936 20948 server_base.cc:1034] running on GCE node
I20260807 15:05:57.821261 20948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:57.821619 20948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:57.822790 20948 hybrid_clock.cc:648] HybridClock initialized: now 1786115157822761 us; error 50 us; skew 500 ppm
I20260807 15:05:57.824263 20948 webserver.cc:460] Webserver started at http://127.1.127.254:34137/ using document root <none> and password file <none>
I20260807 15:05:57.824568 20948 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:57.824651 20948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:57.824813 20948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:57.826000 20948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "952258f74aed4b8baf5ef7c446512385"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "6294bec41bdf9e0cdea5d931357d4cab"
server_key_iv: "4f012d8fdfb7f3f9038da8e7f3b466b2"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.826395 20948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "952258f74aed4b8baf5ef7c446512385"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "6294bec41bdf9e0cdea5d931357d4cab"
server_key_iv: "4f012d8fdfb7f3f9038da8e7f3b466b2"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.827996 20948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:05:57.829067 20962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:57.829336 20948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:57.829483 20948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "952258f74aed4b8baf5ef7c446512385"
format_stamp: "Formatted at 2026-08-07 15:05:57 on dist-test-slave-8tdl"
server_key: "6294bec41bdf9e0cdea5d931357d4cab"
server_key_iv: "4f012d8fdfb7f3f9038da8e7f3b466b2"
server_key_version: "encryptionkey@0"
I20260807 15:05:57.829576 20948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:57.845444 20948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:57.845837 20948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:57.846010 20948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:57.850782 20948 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:39577
I20260807 15:05:57.850832 21014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:39577 every 8 connection(s)
I20260807 15:05:57.851226 20948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:57.852435 21015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:57.855099 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: Bootstrap starting.
I20260807 15:05:57.855743 21015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:57.856112 21015 log.cc:824] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:57.857049 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: No bootstrap required, opened a new log
I20260807 15:05:57.858852 21015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } }
I20260807 15:05:57.859025 21015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:57.859050 21015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 952258f74aed4b8baf5ef7c446512385, State: Initialized, Role: FOLLOWER
I20260807 15:05:57.859247 21015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } }
I20260807 15:05:57.859328 21015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:57.859351 21015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:57.859419 21015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:57.860042 21015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } }
I20260807 15:05:57.860139 21015 leader_election.cc:302] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 952258f74aed4b8baf5ef7c446512385; no voters:
I20260807 15:05:57.860406 21015 leader_election.cc:288] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:05:57.860472 21018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:57.860718 21015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:57.860734 21018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [term 1 LEADER]: Becoming Leader. State: Replica: 952258f74aed4b8baf5ef7c446512385, State: Running, Role: LEADER
I20260807 15:05:57.860826 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 20948
I20260807 15:05:57.860932 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:05:57.860872 21018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } }
I20260807 15:05:57.861161 1535 external_mini_cluster.cc:1428] Setting key 48be94ee31f5b426f48ff31b1f576681
I20260807 15:05:57.861518 21020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 952258f74aed4b8baf5ef7c446512385. Latest consensus state: current_term: 1 leader_uuid: "952258f74aed4b8baf5ef7c446512385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } } }
I20260807 15:05:57.861989 21020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:57.861910 21019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "952258f74aed4b8baf5ef7c446512385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952258f74aed4b8baf5ef7c446512385" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 39577 } } }
I20260807 15:05:57.862133 21019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:57.862380 21036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:57.863205 21036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:57.865046 21036 catalog_manager.cc:1347] Generated new cluster ID: 49482cbc0e5d40359268521648174d1a
I20260807 15:05:57.865108 21036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:57.867786 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:57.861926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.1.127.254:39577)
0807 15:05:57.862616 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:57.862668 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:57.862730 (+ 62us) server_negotiation.cc:407] Connection header received
0807 15:05:57.862855 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:57.862861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:57.862932 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:57.863092 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:57.863379 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.864630 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.865577 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:57.866031 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:57.866173 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:57.866992 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:57.866996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:57.866997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:57.867210 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:57.867247 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:57.867261 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:57.867343 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:57.867422 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 15:05:57.867591 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":83,"threads_started":1}
I20260807 15:05:57.907645 21036 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:05:57.908456 21036 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:57.914747 21036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: Generated new TSK 0
I20260807 15:05:57.914995 21036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:05:57.917603 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--builtin_ntp_servers=127.1.127.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:58.007900 21040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:58.008364 21040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:58.008428 21040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:58.009860 21040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:58.009943 21040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:58.009994 21040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:05:58.011708 21040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:58.012037 21040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:58.012391 21040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:58.017966 21046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.017856 21048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.017903 21045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:58.018285 21040 server_base.cc:1034] running on GCE node
I20260807 15:05:58.018487 21040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:58.018762 21040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:58.019992 21040 hybrid_clock.cc:648] HybridClock initialized: now 1786115158020004 us; error 76 us; skew 500 ppm
I20260807 15:05:58.021816 21040 webserver.cc:460] Webserver started at http://127.1.127.193:40607/ using document root <none> and password file <none>
I20260807 15:05:58.022234 21040 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:58.022333 21040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:58.022528 21040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:58.024008 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "c10b88fff78941c4816946592afe0125"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "2d55fee62903a825cffe5ee9b01d2057"
server_key_iv: "c3223b6008ae49d711520f25edb8cc4e"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.024426 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "c10b88fff78941c4816946592afe0125"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "2d55fee62903a825cffe5ee9b01d2057"
server_key_iv: "c3223b6008ae49d711520f25edb8cc4e"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.026084 21040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:05:58.027172 21055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.027403 21040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:05:58.027549 21040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "c10b88fff78941c4816946592afe0125"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "2d55fee62903a825cffe5ee9b01d2057"
server_key_iv: "c3223b6008ae49d711520f25edb8cc4e"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.027633 21040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:58.046028 21040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:58.046423 21040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:58.046613 21040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:58.046975 21040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:58.047374 21040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:58.047438 21040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.047477 21040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:58.047519 21040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.054615 21040 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:40899
I20260807 15:05:58.054781 21171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:40899 every 8 connection(s)
I20260807 15:05:58.055160 21040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:05:58.061582 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.056108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:38889 (local address 127.1.127.254:39577)
0807 15:05:58.056232 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.056234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.056457 (+ 223us) server_negotiation.cc:407] Connection header received
0807 15:05:58.057169 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.057171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.057187 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.057292 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:58.058100 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.058639 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.059788 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.060016 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.060103 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.061199 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:58.061201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:58.061201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:58.061211 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:58.061234 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:58.061239 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:58.061314 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.061427 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.061482 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 15:05:58.061918 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.056055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39577 (local address 127.1.127.193:38889)
0807 15:05:58.056310 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.056364 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.057051 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.057426 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.057431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.057440 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:58.057981 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.057985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.058830 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.058831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.059677 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.059679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.060114 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.060116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.060290 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.060975 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:58.060976 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:58.061015 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:58.061074 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:58.061316 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:58.061325 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:58.061329 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.061431 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.061600 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260807 15:05:58.062625 21172 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39577
I20260807 15:05:58.062788 21172 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:58.062983 21172 heartbeater.cc:511] Master 127.1.127.254:39577 requested a full tablet report, sending...
I20260807 15:05:58.063666 20979 ts_manager.cc:194] Registered new tserver with Master: c10b88fff78941c4816946592afe0125 (127.1.127.193:40899)
I20260807 15:05:58.064335 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21040
I20260807 15:05:58.064410 20979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.193:38889
I20260807 15:05:58.064464 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:05:58.064776 1535 external_mini_cluster.cc:1428] Setting key 077fd4cc0329820fe5d474c39a370a7d
I20260807 15:05:58.065904 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--builtin_ntp_servers=127.1.127.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:05:58.155001 21175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:58.155275 21175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:58.155313 21175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:58.156546 21175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:58.156631 21175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:58.156674 21175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:05:58.158149 21175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:58.158465 21175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:58.158874 21175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:05:58.164431 21175 server_base.cc:1034] running on GCE node
W20260807 15:05:58.164361 21183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.164371 21181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.164425 21180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:58.165004 21175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:58.165304 21175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:58.166548 21175 hybrid_clock.cc:648] HybridClock initialized: now 1786115158166523 us; error 37 us; skew 500 ppm
I20260807 15:05:58.167990 21175 webserver.cc:460] Webserver started at http://127.1.127.194:40881/ using document root <none> and password file <none>
I20260807 15:05:58.168262 21175 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:58.168434 21175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:58.168589 21175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:58.169796 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "d9b0e64abce6427b846e4615d377957b"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "e8a5abed227217f34c00b8adcf6af1e5"
server_key_iv: "6b475a21210db32d2c779de4c87253a6"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.170174 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "d9b0e64abce6427b846e4615d377957b"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "e8a5abed227217f34c00b8adcf6af1e5"
server_key_iv: "6b475a21210db32d2c779de4c87253a6"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.171742 21175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:05:58.172784 21189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.173054 21175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:58.173166 21175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "d9b0e64abce6427b846e4615d377957b"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "e8a5abed227217f34c00b8adcf6af1e5"
server_key_iv: "6b475a21210db32d2c779de4c87253a6"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.173245 21175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:58.189702 21175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:58.190083 21175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:58.190253 21175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:58.190511 21175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:58.190938 21175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:58.190999 21175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.191041 21175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:58.191084 21175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.197297 21175 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:41179
I20260807 15:05:58.197451 21309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:41179 every 8 connection(s)
I20260807 15:05:58.197805 21175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:05:58.202591 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21175
I20260807 15:05:58.202761 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:05:58.203009 1535 external_mini_cluster.cc:1428] Setting key c28f81c708583dd9662a9287e540dbcf
I20260807 15:05:58.203811 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.198603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:47553 (local address 127.1.127.254:39577)
0807 15:05:58.198650 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.198652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.198961 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:05:58.199453 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.199454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.199468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.199546 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:58.200269 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.200808 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.201944 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.202140 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.202200 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.203508 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:58.203510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:58.203510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:58.203516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:58.203532 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:58.203537 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:58.203592 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.203692 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.203729 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:05:58.204120 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.198542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39577 (local address 127.1.127.194:47553)
0807 15:05:58.198765 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.198809 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.199313 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.199624 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.199628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.199637 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:58.200158 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.200162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.200948 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.200948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.201846 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.201849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.202296 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.202298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.202515 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.203284 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:58.203285 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:58.203316 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:58.203385 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:58.203599 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:58.203604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:58.203608 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.203694 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.203811 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260807 15:05:58.204224 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--builtin_ntp_servers=127.1.127.212:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:05:58.204675 21310 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39577
I20260807 15:05:58.204795 21310 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:58.205047 21310 heartbeater.cc:511] Master 127.1.127.254:39577 requested a full tablet report, sending...
I20260807 15:05:58.205502 20979 ts_manager.cc:194] Registered new tserver with Master: d9b0e64abce6427b846e4615d377957b (127.1.127.194:41179)
I20260807 15:05:58.206089 20979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.194:47553
W20260807 15:05:58.294222 21313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:58.294507 21313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:58.294533 21313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:58.295866 21313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:58.295924 21313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:58.295955 21313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:05:58.297469 21313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:58.297757 21313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:58.298090 21313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:58.303900 21319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.303910 21321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:58.304061 21318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:58.304265 21313 server_base.cc:1034] running on GCE node
I20260807 15:05:58.304553 21313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:58.304807 21313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:58.305971 21313 hybrid_clock.cc:648] HybridClock initialized: now 1786115158305920 us; error 65 us; skew 500 ppm
I20260807 15:05:58.307410 21313 webserver.cc:460] Webserver started at http://127.1.127.195:40785/ using document root <none> and password file <none>
I20260807 15:05:58.307634 21313 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:58.307678 21313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:58.307787 21313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:58.309026 21313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "8250b051bbb74e9598b00fed4bbbd2ea"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "d5cd34e7272b2077617fd7bb1ad0bc5b"
server_key_iv: "a9acfe9674a6b36c04790b2f2f73a8be"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.309357 21313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "8250b051bbb74e9598b00fed4bbbd2ea"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "d5cd34e7272b2077617fd7bb1ad0bc5b"
server_key_iv: "a9acfe9674a6b36c04790b2f2f73a8be"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.310992 21313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:05:58.311848 21328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.312136 21313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:58.312239 21313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "8250b051bbb74e9598b00fed4bbbd2ea"
format_stamp: "Formatted at 2026-08-07 15:05:58 on dist-test-slave-8tdl"
server_key: "d5cd34e7272b2077617fd7bb1ad0bc5b"
server_key_iv: "a9acfe9674a6b36c04790b2f2f73a8be"
server_key_version: "encryptionkey@0"
I20260807 15:05:58.312304 21313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:58.330363 21313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:58.330832 21313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:58.331038 21313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:58.331334 21313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:05:58.331710 21313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:05:58.331746 21313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.331770 21313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:05:58.331818 21313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:58.337952 21313 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:45643
I20260807 15:05:58.338065 21445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:45643 every 8 connection(s)
I20260807 15:05:58.338428 21313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:05:58.341329 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21313
I20260807 15:05:58.341454 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:05:58.341667 1535 external_mini_cluster.cc:1428] Setting key ffe71ecd0d010a5d4b55fd9130fa9671
I20260807 15:05:58.346959 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.339677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:55151 (local address 127.1.127.254:39577)
0807 15:05:58.339752 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.339755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.340063 (+ 308us) server_negotiation.cc:407] Connection header received
0807 15:05:58.341314 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.341317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.341333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.341408 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:58.342530 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.343351 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.344888 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.345158 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.345243 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.346461 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:58.346464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:58.346464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:58.346472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:58.346492 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:58.346498 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:58.346581 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.346812 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.346848 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:05:58.347221 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.339623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:39577 (local address 127.1.127.195:55151)
0807 15:05:58.339897 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.339950 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.340655 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.341621 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.341626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.341639 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:05:58.342360 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.342369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.343500 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.343501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.344767 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.344770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.345277 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.345278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.345460 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.346212 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:05:58.346213 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:05:58.346246 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:05:58.346273 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:05:58.346600 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:05:58.346608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:05:58.346612 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.346813 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.346933 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":115,"threads_started":1}
I20260807 15:05:58.347779 21447 heartbeater.cc:347] Connected to a master server at 127.1.127.254:39577
I20260807 15:05:58.347886 21447 heartbeater.cc:464] Registering TS with master...
I20260807 15:05:58.348138 21447 heartbeater.cc:511] Master 127.1.127.254:39577 requested a full tablet report, sending...
I20260807 15:05:58.348735 20979 ts_manager.cc:194] Registered new tserver with Master: 8250b051bbb74e9598b00fed4bbbd2ea (127.1.127.195:45643)
I20260807 15:05:58.349192 20979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.195:55151
I20260807 15:05:58.355581 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:05:58.360677 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.357271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.1.127.254:39577)
0807 15:05:58.357341 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.357343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.357491 (+ 148us) server_negotiation.cc:407] Connection header received
0807 15:05:58.357547 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.357547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.357559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.357618 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:58.357763 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.358264 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.359252 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.359447 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.359517 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.360297 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:58.360298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:58.360298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:05:58.360303 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:58.360319 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:05:58.360339 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:58.360468 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.360579 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.360593 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:05:58.362901 20979 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:05:58.364575 20979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:05:58.371199 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.367147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41179 (local address 127.0.0.1:45164)
0807 15:05:58.367484 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.367495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.367541 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.367778 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.367779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.367781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:58.367817 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.367819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.369158 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.369159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.369912 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.369914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.370868 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.370869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.370950 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.370965 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.371046 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.371098 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":52,"threads_started":1}
I20260807 15:05:58.371215 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.367284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.1.127.194:41179)
0807 15:05:58.367520 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.367525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.367535 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:05:58.367614 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.367616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.367633 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.367709 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:58.367897 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.369057 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.370022 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.370772 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.370852 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.370950 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.371078 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.371125 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260807 15:05:58.371613 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.367281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.1.127.195:45643)
0807 15:05:58.367456 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.367461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.367473 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:05:58.367519 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.367521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.367538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.367844 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:58.368034 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.368831 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.370271 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.371042 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.371114 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.371197 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.371482 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.371524 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260807 15:05:58.371560 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.366877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:45643 (local address 127.0.0.1:34308)
0807 15:05:58.367030 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.367074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.367164 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.367919 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.367920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.367922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:58.367971 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.367973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.368934 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.368936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.370179 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.370182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.371236 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.371237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.371358 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.371362 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.371445 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.371490 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260807 15:05:58.371856 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.366878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40899 (local address 127.0.0.1:54984)
0807 15:05:58.367237 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.367249 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.367323 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.367663 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.367666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.367669 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:58.367755 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.367758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.369140 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.369141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.370474 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.370476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.371582 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.371583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.371660 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.371664 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.371727 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.371773 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":5,"thread_start_us":132,"threads_started":1}
I20260807 15:05:58.372366 21379 tablet_service.cc:1467] Processing CreateTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 (DEFAULT_TABLE table=test-table [id=f635232b0c824e3ebaa6b4ad7e817e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:58.372334 21240 tablet_service.cc:1467] Processing CreateTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 (DEFAULT_TABLE table=test-table [id=f635232b0c824e3ebaa6b4ad7e817e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:58.372766 21379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c76c11f0ff34d06bbf61adf2567ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:58.372820 21240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c76c11f0ff34d06bbf61adf2567ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:58.374917 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.366955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.1.127.193:40899)
0807 15:05:58.367162 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.367167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.367336 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:05:58.367394 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.367397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.367415 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.367536 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:58.367847 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.369047 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.370558 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.371399 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.371468 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.371548 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.371749 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.374822 (+ 3073us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260807 15:05:58.375658 21104 tablet_service.cc:1467] Processing CreateTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 (DEFAULT_TABLE table=test-table [id=f635232b0c824e3ebaa6b4ad7e817e86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:05:58.376045 21104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c76c11f0ff34d06bbf61adf2567ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:58.376044 21461 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Bootstrap starting.
I20260807 15:05:58.376785 21461 tablet_bootstrap.cc:654] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:58.377122 21461 log.cc:824] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:58.377394 21462 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Bootstrap starting.
I20260807 15:05:58.378000 21461 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: No bootstrap required, opened a new log
I20260807 15:05:58.378103 21462 tablet_bootstrap.cc:654] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:58.378166 21461 ts_tablet_manager.cc:1397] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:05:58.378588 21462 log.cc:824] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:58.379444 21466 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: Bootstrap starting.
I20260807 15:05:58.379535 21462 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: No bootstrap required, opened a new log
I20260807 15:05:58.379622 21462 ts_tablet_manager.cc:1397] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 15:05:58.380155 21466 tablet_bootstrap.cc:654] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:58.380489 21466 log.cc:824] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:58.381250 21466 tablet_bootstrap.cc:492] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: No bootstrap required, opened a new log
I20260807 15:05:58.381181 21461 raft_consensus.cc:348] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.381343 21466 ts_tablet_manager.cc:1397] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:05:58.381371 21461 raft_consensus.cc:374] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:58.381417 21461 raft_consensus.cc:729] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8250b051bbb74e9598b00fed4bbbd2ea, State: Initialized, Role: FOLLOWER
I20260807 15:05:58.381575 21461 consensus_queue.cc:260] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.381887 21461 ts_tablet_manager.cc:1428] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 15:05:58.382045 21447 heartbeater.cc:503] Master 127.1.127.254:39577 was elected leader, sending a full tablet report...
I20260807 15:05:58.382472 21462 raft_consensus.cc:348] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.382660 21462 raft_consensus.cc:374] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:58.382752 21462 raft_consensus.cc:729] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b0e64abce6427b846e4615d377957b, State: Initialized, Role: FOLLOWER
I20260807 15:05:58.382887 21462 consensus_queue.cc:260] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.383174 21462 ts_tablet_manager.cc:1428] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:05:58.383255 21310 heartbeater.cc:503] Master 127.1.127.254:39577 was elected leader, sending a full tablet report...
I20260807 15:05:58.384595 21466 raft_consensus.cc:348] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.384758 21466 raft_consensus.cc:374] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:58.384805 21466 raft_consensus.cc:729] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10b88fff78941c4816946592afe0125, State: Initialized, Role: FOLLOWER
I20260807 15:05:58.384938 21466 consensus_queue.cc:260] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.385245 21466 ts_tablet_manager.cc:1428] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:05:58.385303 21172 heartbeater.cc:503] Master 127.1.127.254:39577 was elected leader, sending a full tablet report...
W20260807 15:05:58.448475 21311 tablet.cc:2369] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:05:58.485639 21471 raft_consensus.cc:482] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:05:58.485803 21471 raft_consensus.cc:504] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.486263 21471 leader_election.cc:288] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8250b051bbb74e9598b00fed4bbbd2ea (127.1.127.195:45643), d9b0e64abce6427b846e4615d377957b (127.1.127.194:41179)
I20260807 15:05:58.490144 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.486714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:34813 (local address 127.1.127.195:45643)
0807 15:05:58.486780 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.486782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.486798 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:05:58.486909 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.486910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.486922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.487032 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:58.487222 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.488001 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.489198 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.489755 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.489819 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.489827 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.490041 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.490082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:05:58.490159 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.486582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:45643 (local address 127.1.127.193:34813)
0807 15:05:58.486669 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.486712 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.486790 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.487038 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.487039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.487040 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:58.487132 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.487134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.488092 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.488093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.489079 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.489082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.489866 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.489867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.489947 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.489952 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.490021 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.490073 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:05:58.490566 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.486848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41179 (local address 127.1.127.193:41171)
0807 15:05:58.487178 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.487189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:05:58.487252 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:05:58.487424 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:05:58.487425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:05:58.487427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:05:58.487503 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.487505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.488334 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.488335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.489470 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:05:58.489474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.490317 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.490318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:05:58.490385 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.490390 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:05:58.490443 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:05:58.490482 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":248,"threads_started":1}
I20260807 15:05:58.490581 21400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c76c11f0ff34d06bbf61adf2567ec27" candidate_uuid: "c10b88fff78941c4816946592afe0125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" is_pre_election: true
I20260807 15:05:58.490778 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.486866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:41171 (local address 127.1.127.194:41179)
0807 15:05:58.486927 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.486929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.487244 (+ 315us) server_negotiation.cc:407] Connection header received
0807 15:05:58.487312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.487313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.487325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.487376 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:05:58.487566 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.488242 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.489606 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.490186 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.490260 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.490270 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.490636 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.490688 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:05:58.490799 21400 raft_consensus.cc:2454] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c10b88fff78941c4816946592afe0125 in term 0.
I20260807 15:05:58.491071 21057 leader_election.cc:302] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8250b051bbb74e9598b00fed4bbbd2ea, c10b88fff78941c4816946592afe0125; no voters:
I20260807 15:05:58.491235 21471 raft_consensus.cc:2781] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:05:58.491214 21262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c76c11f0ff34d06bbf61adf2567ec27" candidate_uuid: "c10b88fff78941c4816946592afe0125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b0e64abce6427b846e4615d377957b" is_pre_election: true
I20260807 15:05:58.491304 21471 raft_consensus.cc:482] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:05:58.491336 21471 raft_consensus.cc:3037] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:58.491393 21262 raft_consensus.cc:2454] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c10b88fff78941c4816946592afe0125 in term 0.
I20260807 15:05:58.492139 21471 raft_consensus.cc:504] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.492295 21471 leader_election.cc:288] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [CANDIDATE]: Term 1 election: Requested vote from peers 8250b051bbb74e9598b00fed4bbbd2ea (127.1.127.195:45643), d9b0e64abce6427b846e4615d377957b (127.1.127.194:41179)
I20260807 15:05:58.492570 21400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c76c11f0ff34d06bbf61adf2567ec27" candidate_uuid: "c10b88fff78941c4816946592afe0125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8250b051bbb74e9598b00fed4bbbd2ea"
I20260807 15:05:58.492583 21262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c76c11f0ff34d06bbf61adf2567ec27" candidate_uuid: "c10b88fff78941c4816946592afe0125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b0e64abce6427b846e4615d377957b"
I20260807 15:05:58.492681 21400 raft_consensus.cc:3037] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:58.492673 21262 raft_consensus.cc:3037] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:58.493492 21262 raft_consensus.cc:2454] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c10b88fff78941c4816946592afe0125 in term 1.
I20260807 15:05:58.493501 21400 raft_consensus.cc:2454] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c10b88fff78941c4816946592afe0125 in term 1.
I20260807 15:05:58.493673 21057 leader_election.cc:302] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8250b051bbb74e9598b00fed4bbbd2ea, c10b88fff78941c4816946592afe0125; no voters:
I20260807 15:05:58.493788 21471 raft_consensus.cc:2781] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:58.493983 21471 raft_consensus.cc:686] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [term 1 LEADER]: Becoming Leader. State: Replica: c10b88fff78941c4816946592afe0125, State: Running, Role: LEADER
I20260807 15:05:58.494097 21471 consensus_queue.cc:237] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } }
I20260807 15:05:58.495080 20979 catalog_manager.cc:5495] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 reported cstate change: term changed from 0 to 1, leader changed from <none> to c10b88fff78941c4816946592afe0125 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "c10b88fff78941c4816946592afe0125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c10b88fff78941c4816946592afe0125" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 40899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 } health_report { overall_health: UNKNOWN } } }
I20260807 15:05:58.522161 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.519562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.1.127.193:40899)
0807 15:05:58.519636 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:58.519638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:58.519647 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:05:58.519686 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:58.519687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:58.519708 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:58.519772 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:05:58.519908 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.520435 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.521292 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:58.521498 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:58.521565 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:58.521709 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:05:58.521802 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:05:58.521874 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:58.522022 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:05:58.522038 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:05:58.524310 21400 raft_consensus.cc:1261] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 1 FOLLOWER]: Refusing update from remote peer c10b88fff78941c4816946592afe0125: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:58.524310 21262 raft_consensus.cc:1261] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 1 FOLLOWER]: Refusing update from remote peer c10b88fff78941c4816946592afe0125: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:05:58.524765 21471 consensus_queue.cc:1035] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8250b051bbb74e9598b00fed4bbbd2ea" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 45643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:58.525007 21473 consensus_queue.cc:1035] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9b0e64abce6427b846e4615d377957b" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:05:58.528093 21478 mvcc.cc:205] Tried to move back new op lower bound from 7315927689312694272 to 7315927689192652800. Current Snapshot: MvccSnapshot[applied={T|T < 7315927689312694272}]
I20260807 15:05:58.528396 21477 mvcc.cc:205] Tried to move back new op lower bound from 7315927689312694272 to 7315927689192652800. Current Snapshot: MvccSnapshot[applied={T|T < 7315927689312694272}]
I20260807 15:05:58.530179 20979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:58.530301 20979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:05:58.531186 20979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 952258f74aed4b8baf5ef7c446512385: Sending DeleteTablet for 3 replicas of tablet 4c76c11f0ff34d06bbf61adf2567ec27
I20260807 15:05:58.531530 21104 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:58 UTC) from {username='slave'} at 127.0.0.1:54984
I20260807 15:05:58.531672 21240 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:58 UTC) from {username='slave'} at 127.0.0.1:45164
I20260807 15:05:58.531854 21487 tablet_replica.cc:330] T 4c76c11f0ff34d06bbf61adf2567ec27 P c10b88fff78941c4816946592afe0125: stopping tablet replica
I20260807 15:05:58.531899 21379 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c76c11f0ff34d06bbf61adf2567ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:05:58 UTC) from {username='slave'} at 127.0.0.1:34308
I20260807 15:05:58.531966 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21040
I20260807 15:05:58.538897 21488 tablet_replica.cc:330] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: stopping tablet replica
I20260807 15:05:58.539114 21488 raft_consensus.cc:2229] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:05:58.539216 21489 tablet_replica.cc:330] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: stopping tablet replica
I20260807 15:05:58.539275 21488 raft_consensus.cc:2258] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:58.539362 21489 raft_consensus.cc:2229] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 15:05:58.539521 20965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:40899 (error 108)
I20260807 15:05:58.539705 21489 raft_consensus.cc:2258] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:05:58.539836 21488 ts_tablet_manager.cc:1905] T 4c76c11f0ff34d06bbf61adf2567ec27 P d9b0e64abce6427b846e4615d377957b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:58.540007 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21175
I20260807 15:05:58.540606 21489 ts_tablet_manager.cc:1905] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:05:58.541589 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.540055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:40899 (local address 127.0.0.1:55002)
0807 15:05:58.541465 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.541545 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1377}
W20260807 15:05:58.541760 20965 catalog_manager.cc:4584] TS c10b88fff78941c4816946592afe0125 (127.1.127.193:40899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c76c11f0ff34d06bbf61adf2567ec27: Network error: Client connection negotiation failed: client connection to 127.1.127.193:40899: connect: Connection refused (error 111)
I20260807 15:05:58.542878 21489 ts_tablet_manager.cc:1918] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:05:58.542971 21489 log.cc:1196] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/4c76c11f0ff34d06bbf61adf2567ec27
I20260807 15:05:58.543439 21489 ts_tablet_manager.cc:1939] T 4c76c11f0ff34d06bbf61adf2567ec27 P 8250b051bbb74e9598b00fed4bbbd2ea: Deleting consensus metadata
I20260807 15:05:58.544071 20963 catalog_manager.cc:4841] TS 8250b051bbb74e9598b00fed4bbbd2ea (127.1.127.195:45643): tablet 4c76c11f0ff34d06bbf61adf2567ec27 (table test-table [id=f635232b0c824e3ebaa6b4ad7e817e86]) successfully deleted
I20260807 15:05:58.547833 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21313
I20260807 15:05:58.548142 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:58.547898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:41179 (local address 127.0.0.1:45166)
0807 15:05:58.548025 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 15:05:58.548095 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.194:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 15:05:58.553637 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 20948
2026-08-07T15:05:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (863 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:05:58 dist-test-slave-8tdl krb5kdc[21493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:05:58 dist-test-slave-8tdl krb5kdc[21493](info): set up 2 sockets
Aug 07 15:05:58 dist-test-slave-8tdl krb5kdc[21493](info): commencing operation
krb5kdc: starting...
W20260807 15:05:59.648520 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:05:59 dist-test-slave-8tdl krb5kdc[21493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:05:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:05:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:05:59.804881 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35459
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:35459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:05:59.896505 21506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:05:59.896782 21506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:05:59.896811 21506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:05:59.898278 21506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:05:59.898348 21506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:05:59.898366 21506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:05:59.898381 21506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:05:59.898396 21506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:05:59.900390 21506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:35459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:35459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:05:59.900723 21506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:05:59.901053 21506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:05:59.907162 21514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:59.907236 21516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:05:59.907189 21513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:05:59.907541 21506 server_base.cc:1034] running on GCE node
I20260807 15:05:59.907804 21506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:05:59.908166 21506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:05:59.909373 21506 hybrid_clock.cc:648] HybridClock initialized: now 1786115159909359 us; error 35 us; skew 500 ppm
Aug 07 15:05:59 dist-test-slave-8tdl krb5kdc[21493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:05:59.913246 21506 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:05:59.913942 21506 webserver.cc:460] Webserver started at http://127.1.127.254:39059/ using document root <none> and password file <none>
I20260807 15:05:59.914248 21506 fs_manager.cc:359] Metadata directory not provided
I20260807 15:05:59.914342 21506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:05:59.914551 21506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:05:59.915999 21506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "0ce07a48502744dc89743b3a5ff21bf0"
format_stamp: "Formatted at 2026-08-07 15:05:59 on dist-test-slave-8tdl"
server_key: "81021b5d4664487d2ee27d080bf2ccf6"
server_key_iv: "71a51d8a2243a64f7b0c51e5f2714bc8"
server_key_version: "encryptionkey@0"
I20260807 15:05:59.916628 21506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "0ce07a48502744dc89743b3a5ff21bf0"
format_stamp: "Formatted at 2026-08-07 15:05:59 on dist-test-slave-8tdl"
server_key: "81021b5d4664487d2ee27d080bf2ccf6"
server_key_iv: "71a51d8a2243a64f7b0c51e5f2714bc8"
server_key_version: "encryptionkey@0"
I20260807 15:05:59.918363 21506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:05:59.919449 21523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:05:59.919790 21506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:05:59.919904 21506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "0ce07a48502744dc89743b3a5ff21bf0"
format_stamp: "Formatted at 2026-08-07 15:05:59 on dist-test-slave-8tdl"
server_key: "81021b5d4664487d2ee27d080bf2ccf6"
server_key_iv: "71a51d8a2243a64f7b0c51e5f2714bc8"
server_key_version: "encryptionkey@0"
I20260807 15:05:59.920049 21506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:05:59.930976 21506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:05:59.933560 21506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:05:59.933763 21506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:05:59.939342 21506 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:35459
I20260807 15:05:59.939416 21576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:35459 every 8 connection(s)
I20260807 15:05:59.939836 21506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:05:59.940907 21577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:05:59.941927 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21506
I20260807 15:05:59.942101 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:05:59.942319 1535 external_mini_cluster.cc:1428] Setting key ab2831776c4e625704c8572221d8e6dc
I20260807 15:05:59.944094 21577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: Bootstrap starting.
I20260807 15:05:59.944852 21577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: Neither blocks nor log segments found. Creating new log.
I20260807 15:05:59.945309 21577 log.cc:824] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:05:59.946331 21577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: No bootstrap required, opened a new log
I20260807 15:05:59.948484 21577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } }
I20260807 15:05:59.948678 21577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:05:59.948720 21577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce07a48502744dc89743b3a5ff21bf0, State: Initialized, Role: FOLLOWER
I20260807 15:05:59.948931 21577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } }
I20260807 15:05:59.949040 21577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:05:59.949091 21577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:05:59.949164 21577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:05:59.949993 21577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } }
I20260807 15:05:59.950133 21577 leader_election.cc:302] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ce07a48502744dc89743b3a5ff21bf0; no voters:
I20260807 15:05:59.950349 21577 leader_election.cc:288] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 07 15:05:59 dist-test-slave-8tdl krb5kdc[21493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:05:59.950428 21582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:05:59.950659 21582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [term 1 LEADER]: Becoming Leader. State: Replica: 0ce07a48502744dc89743b3a5ff21bf0, State: Running, Role: LEADER
I20260807 15:05:59.950711 21577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:05:59.950806 21582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } }
I20260807 15:05:59.951182 21582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ce07a48502744dc89743b3a5ff21bf0. Latest consensus state: current_term: 1 leader_uuid: "0ce07a48502744dc89743b3a5ff21bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } } }
I20260807 15:05:59.951280 21582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:59.951195 21584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ce07a48502744dc89743b3a5ff21bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce07a48502744dc89743b3a5ff21bf0" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 35459 } } }
I20260807 15:05:59.951473 21584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:05:59.951988 21589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:05:59.953153 21589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:05:59.955394 21589 catalog_manager.cc:1347] Generated new cluster ID: 48455ae1aaa64bb58bf4a38e7bbaec08
I20260807 15:05:59.955452 21589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:05:59.955732 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:05:59.943076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.1.127.254:35459)
0807 15:05:59.943284 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 15:05:59.943296 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:05:59.943453 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:05:59.943995 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:05:59.944002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:05:59.944061 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:05:59.944236 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:05:59.945024 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:59.946285 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:59.947183 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:05:59.947660 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:05:59.947796 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:05:59.951124 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:05:59.951128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:05:59.951129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:05:59.951144 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:05:59.953670 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:59.954100 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:59.954104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:59.954105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:59.954228 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:05:59.954525 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:05:59.954527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:05:59.954527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:05:59.954895 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:05:59.955064 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 15:05:59.955218 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:05:59.955372 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260807 15:05:59.980253 21589 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:05:59.980922 21589 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:05:59.984112 21589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: Generated new TSK 0
I20260807 15:05:59.984345 21589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.127.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
Entry for principal kudu/127.1.127.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab.
I20260807 15:06:00.036512 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:00.128330 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:00.128614 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:00.128659 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:00.129997 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:00.130082 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:00.130131 21604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:06:00.131786 21604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.193.keytab
--principal=kudu/127.1.127.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:00.132084 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:00.132342 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:00.138134 21609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.138134 21610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.138190 21612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:00.138625 21604 server_base.cc:1034] running on GCE node
I20260807 15:06:00.138890 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:00.139173 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:00.140451 21604 hybrid_clock.cc:648] HybridClock initialized: now 1786115160140433 us; error 33 us; skew 500 ppm
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:00.144150 21604 init.cc:377] Logged in from keytab as kudu/127.1.127.193@KRBTEST.COM (short username kudu)
I20260807 15:06:00.144790 21604 webserver.cc:460] Webserver started at http://127.1.127.193:44167/ using document root <none> and password file <none>
I20260807 15:06:00.145044 21604 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:00.145093 21604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:00.145205 21604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:00.146577 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "7e0ae90692cf4421aac7af98529f1fc8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9a0e6899cf4427f22ea6a485c4747789"
server_key_iv: "a4d8544ebf34562d2fde49d32fe7fdd0"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.147058 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "7e0ae90692cf4421aac7af98529f1fc8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9a0e6899cf4427f22ea6a485c4747789"
server_key_iv: "a4d8544ebf34562d2fde49d32fe7fdd0"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.149226 21604 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:06:00.150648 21620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.151015 21604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:00.151139 21604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "7e0ae90692cf4421aac7af98529f1fc8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9a0e6899cf4427f22ea6a485c4747789"
server_key_iv: "a4d8544ebf34562d2fde49d32fe7fdd0"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.151283 21604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:00.163621 21604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:00.165900 21604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:00.166077 21604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:00.166333 21604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:00.166762 21604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:00.166821 21604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.166867 21604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:00.166893 21604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.173364 21604 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:43493
I20260807 15:06:00.173481 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:43493 every 8 connection(s)
I20260807 15:06:00.173957 21604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.193@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:00.183459 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21604
I20260807 15:06:00.183588 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:00.183820 1535 external_mini_cluster.cc:1428] Setting key b02442b3e56e0dd8048c8eafee5e5da3
I20260807 15:06:00.186364 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.175369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:41861 (local address 127.1.127.254:35459)
0807 15:06:00.175459 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.175461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.175673 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:06:00.176246 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.176255 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.176267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.176359 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:00.177701 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.178262 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.179486 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.179704 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.179768 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.182596 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:00.182598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:00.182599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:00.182608 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:00.184690 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.185196 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.185197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.185198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.185282 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.185592 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.185593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.185594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.185733 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:00.185809 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.186138 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.186260 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:06:00.186628 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.175311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35459 (local address 127.1.127.193:41861)
0807 15:06:00.175527 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.175575 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.176090 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.176592 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.176596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.177280 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:00.177500 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.177507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.178375 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.178376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.179362 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.179366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.179800 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.179801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.179943 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.180667 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:00.180677 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:00.182291 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:00.184771 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.184785 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.184788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.185094 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.185359 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.185360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.185360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.185510 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.185814 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:00.185824 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:00.185990 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.186118 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.186327 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260807 15:06:00.187331 21739 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35459
I20260807 15:06:00.187449 21739 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:00.187677 21739 heartbeater.cc:511] Master 127.1.127.254:35459 requested a full tablet report, sending...
I20260807 15:06:00.188411 21540 ts_manager.cc:194] Registered new tserver with Master: 7e0ae90692cf4421aac7af98529f1fc8 (127.1.127.193:43493)
I20260807 15:06:00.189163 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.193@KRBTEST.COM'} at 127.1.127.193:41861
WARNING: no policy specified for kudu/127.1.127.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
Entry for principal kudu/127.1.127.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab.
I20260807 15:06:00.220150 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:00.317116 21745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:00.317453 21745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:00.317520 21745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:00.319115 21745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:00.319218 21745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:00.319278 21745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:06:00.321264 21745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.194.keytab
--principal=kudu/127.1.127.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:00.321627 21745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:00.321947 21745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:00.329746 21751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.329768 21750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.329754 21753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:00.329810 21745 server_base.cc:1034] running on GCE node
I20260807 15:06:00.330265 21745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:00.330602 21745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:00.331810 21745 hybrid_clock.cc:648] HybridClock initialized: now 1786115160331746 us; error 79 us; skew 500 ppm
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:00.335510 21745 init.cc:377] Logged in from keytab as kudu/127.1.127.194@KRBTEST.COM (short username kudu)
I20260807 15:06:00.336046 21745 webserver.cc:460] Webserver started at http://127.1.127.194:42991/ using document root <none> and password file <none>
I20260807 15:06:00.336277 21745 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:00.336331 21745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:00.336429 21745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:00.337549 21745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "9ab56d7a256c41258b15d9e3e739ae20"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "06234fa771ed0d1dd0856ad2d26bd6e5"
server_key_iv: "48a8a81215118fcc908c7d11d9def878"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.337909 21745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "9ab56d7a256c41258b15d9e3e739ae20"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "06234fa771ed0d1dd0856ad2d26bd6e5"
server_key_iv: "48a8a81215118fcc908c7d11d9def878"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.339551 21745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:00.340623 21760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.340862 21745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:00.340966 21745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "9ab56d7a256c41258b15d9e3e739ae20"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "06234fa771ed0d1dd0856ad2d26bd6e5"
server_key_iv: "48a8a81215118fcc908c7d11d9def878"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.341049 21745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:00.377893 21745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:00.380808 21745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:00.380955 21745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:00.381258 21745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:00.381652 21745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:00.381693 21745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.381716 21745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:00.381731 21745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.388854 21745 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:35741
I20260807 15:06:00.388904 21879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:35741 every 8 connection(s)
I20260807 15:06:00.389261 21745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.194@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:00.398204 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21745
I20260807 15:06:00.398311 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:00.398545 1535 external_mini_cluster.cc:1428] Setting key 2c09658d5bc72737faaf40f8f841fccf
I20260807 15:06:00.400302 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.390099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:38453 (local address 127.1.127.254:35459)
0807 15:06:00.390164 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.390167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.390521 (+ 354us) server_negotiation.cc:407] Connection header received
0807 15:06:00.391288 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.391289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.391302 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.391375 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:00.392565 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.393153 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.394204 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.394422 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.394501 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.396958 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:00.396960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:00.396960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:00.396968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:00.398637 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.399205 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.399207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.399207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.399272 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.399594 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.399595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.399595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.399720 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:00.399795 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.400114 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.400207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:06:00.400535 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.390130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35459 (local address 127.1.127.194:38453)
0807 15:06:00.390335 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.390376 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.390985 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.391556 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.391560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.392218 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:00.392436 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.392441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.393255 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.393256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.394092 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.394095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.394536 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.394537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.394651 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.395239 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:00.395248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:00.396828 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:00.398789 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.398798 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.398801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.399106 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.399353 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.399354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.399354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.399526 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.399798 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:00.399806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:00.399969 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.400080 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.400267 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260807 15:06:00.401067 21880 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35459
I20260807 15:06:00.401198 21880 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:00.401459 21880 heartbeater.cc:511] Master 127.1.127.254:35459 requested a full tablet report, sending...
I20260807 15:06:00.401962 21540 ts_manager.cc:194] Registered new tserver with Master: 9ab56d7a256c41258b15d9e3e739ae20 (127.1.127.194:35741)
I20260807 15:06:00.402417 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.194@KRBTEST.COM'} at 127.1.127.194:38453
WARNING: no policy specified for kudu/127.1.127.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
Entry for principal kudu/127.1.127.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab.
I20260807 15:06:00.431422 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:00.517191 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:00.517416 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:00.517442 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:00.518656 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:00.518764 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:00.518795 21885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:06:00.520345 21885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.195.keytab
--principal=kudu/127.1.127.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:35459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:00.520627 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:00.520902 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:00.526585 21890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.526585 21891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.526602 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:00.527015 21885 server_base.cc:1034] running on GCE node
I20260807 15:06:00.527262 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:00.527513 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:00.528690 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786115160528672 us; error 35 us; skew 500 ppm
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:00.531876 21885 init.cc:377] Logged in from keytab as kudu/127.1.127.195@KRBTEST.COM (short username kudu)
I20260807 15:06:00.532416 21885 webserver.cc:460] Webserver started at http://127.1.127.195:34233/ using document root <none> and password file <none>
I20260807 15:06:00.532892 21885 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:00.532996 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:00.533205 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:00.534446 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "fe67dfbc24c14092a0b83ddb5d4171aa"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "51bd527b97d4dfa890c5b91290fc199c"
server_key_iv: "5f0cd368c7679f9bd20ed488af394223"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.535008 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "fe67dfbc24c14092a0b83ddb5d4171aa"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "51bd527b97d4dfa890c5b91290fc199c"
server_key_iv: "5f0cd368c7679f9bd20ed488af394223"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.536782 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 15:06:00.537984 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.538254 21885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:00.538368 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "fe67dfbc24c14092a0b83ddb5d4171aa"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "51bd527b97d4dfa890c5b91290fc199c"
server_key_iv: "5f0cd368c7679f9bd20ed488af394223"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.538465 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:00.555464 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:00.558090 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:00.558300 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:00.558607 21885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:00.559026 21885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:00.559090 21885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.559140 21885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:00.559176 21885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.566103 21885 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:39575
I20260807 15:06:00.566263 22022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:39575 every 8 connection(s)
I20260807 15:06:00.566584 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:00.567749 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 21885
I20260807 15:06:00.567898 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:00.568094 1535 external_mini_cluster.cc:1428] Setting key 7b977851bdfef582baef9338bad633b6
Aug 07 15:06:00 dist-test-slave-8tdl krb5kdc[21493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.195@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:00.578389 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.567508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:37749 (local address 127.1.127.254:35459)
0807 15:06:00.567598 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.567600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.567808 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:06:00.568575 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.568576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.568604 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.568669 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:00.569875 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.570483 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.571602 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.571856 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.571927 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.574932 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:00.574934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:00.574934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:00.574942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:00.576765 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.577311 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.577313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.577313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.577390 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.577696 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.577697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.577697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.577838 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:00.577904 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.578229 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.578309 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:06:00.578574 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.567455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:35459 (local address 127.1.127.195:37749)
0807 15:06:00.567666 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.567714 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.568438 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.568751 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.568756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.569516 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:00.569725 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.569730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.570595 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.570596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.571459 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.571461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.572127 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.572128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.572282 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.572997 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:00.573006 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:00.574785 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:00.576863 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.576873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.576875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.577209 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.577474 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:00.577475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:00.577476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:00.577605 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:00.577902 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:00.577906 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:00.578090 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.578183 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.578328 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260807 15:06:00.579185 22023 heartbeater.cc:347] Connected to a master server at 127.1.127.254:35459
I20260807 15:06:00.579298 22023 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:00.579551 22023 heartbeater.cc:511] Master 127.1.127.254:35459 requested a full tablet report, sending...
I20260807 15:06:00.580017 21540 ts_manager.cc:194] Registered new tserver with Master: fe67dfbc24c14092a0b83ddb5d4171aa (127.1.127.195:39575)
I20260807 15:06:00.580466 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.127.195@KRBTEST.COM'} at 127.1.127.195:37749
I20260807 15:06:00.591928 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:00.601012 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.593681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.1.127.254:35459)
0807 15:06:00.593763 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.593766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.593778 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:00.593806 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.593812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.593822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.593896 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:00.594628 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.595333 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.596195 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.596418 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.596519 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.597962 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:00.597964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:00.597964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:00.597978 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:00.599607 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.600048 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.600049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.600049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.600118 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:00.600333 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:00.600334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:00.600335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:00.600526 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:00.600642 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.600788 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.600900 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:06:00.603404 21540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:00.605238 21540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:00.612468 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.1.127.194:35741)
0807 15:06:00.608616 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.608621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.608633 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:00.608678 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.608681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.608698 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.608797 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:00.609098 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.609906 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.611265 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612040 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.612129 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.612200 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.612322 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.612372 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260807 15:06:00.612455 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:35741 (local address 127.0.0.1:54738)
0807 15:06:00.608370 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.608413 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.608481 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.608823 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.608826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.608829 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:00.608993 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.608996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.610021 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.610022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.611173 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.611176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612132 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.612133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.612215 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.612229 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.612307 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.612367 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
I20260807 15:06:00.612874 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:39575 (local address 127.0.0.1:51400)
0807 15:06:00.608548 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.608561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.608628 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.609188 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.609189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.609192 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:00.609254 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.609255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.610579 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.610580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.611558 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.611560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612394 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.612395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.612460 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.612465 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.612498 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.612541 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":50,"thread_start_us":74,"threads_started":1}
I20260807 15:06:00.613104 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.1.127.193:43493)
0807 15:06:00.608939 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.608944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.608955 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:00.608994 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.608996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.609013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.609088 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:00.609306 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.610473 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.611958 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612693 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.612758 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.612808 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.612969 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.613023 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260807 15:06:00.613131 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:43493 (local address 127.0.0.1:38348)
0807 15:06:00.608731 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.608741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.608778 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.609183 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.609184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.609186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:00.609235 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.609237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.610607 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.610608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.611861 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.611864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612792 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.612793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.612870 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.612874 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.612936 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.613064 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260807 15:06:00.613564 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 (DEFAULT_TABLE table=test-table [id=dbb70d68068540739c13a614ec8d49b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:00.614006 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365d846e967c491e88c2b3bf4557a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:00.614041 21672 tablet_service.cc:1467] Processing CreateTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 (DEFAULT_TABLE table=test-table [id=dbb70d68068540739c13a614ec8d49b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:00.614430 21672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365d846e967c491e88c2b3bf4557a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:00.616698 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.608325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51400 (local address 127.1.127.195:39575)
0807 15:06:00.609015 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.609019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.609030 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:00.609068 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.609070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.609085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.609333 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:00.609413 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.610458 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.611651 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.612290 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.613238 (+ 948us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.613400 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.613439 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.616579 (+ 3140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":160,"threads_started":1}
I20260807 15:06:00.616935 22040 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Bootstrap starting.
I20260807 15:06:00.617435 22041 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Bootstrap starting.
I20260807 15:06:00.617429 21951 tablet_service.cc:1467] Processing CreateTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 (DEFAULT_TABLE table=test-table [id=dbb70d68068540739c13a614ec8d49b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:00.617676 22040 tablet_bootstrap.cc:654] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:00.617790 21951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365d846e967c491e88c2b3bf4557a5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:00.618233 22041 tablet_bootstrap.cc:654] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:00.618296 22040 log.cc:824] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:00.618978 22041 log.cc:824] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:00.619277 22040 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: No bootstrap required, opened a new log
I20260807 15:06:00.619365 22040 ts_tablet_manager.cc:1397] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:00.619733 22041 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: No bootstrap required, opened a new log
I20260807 15:06:00.619823 22041 ts_tablet_manager.cc:1397] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:06:00.621479 22044 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Bootstrap starting.
I20260807 15:06:00.621572 22040 raft_consensus.cc:348] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.621600 22041 raft_consensus.cc:348] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.621799 22041 raft_consensus.cc:374] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:00.621801 22040 raft_consensus.cc:374] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:00.621858 22040 raft_consensus.cc:729] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab56d7a256c41258b15d9e3e739ae20, State: Initialized, Role: FOLLOWER
I20260807 15:06:00.621855 22041 raft_consensus.cc:729] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0ae90692cf4421aac7af98529f1fc8, State: Initialized, Role: FOLLOWER
I20260807 15:06:00.621992 22041 consensus_queue.cc:260] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.622006 22040 consensus_queue.cc:260] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.622320 22040 ts_tablet_manager.cc:1428] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:06:00.622323 22041 ts_tablet_manager.cc:1428] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:06:00.622372 22044 tablet_bootstrap.cc:654] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:00.622499 21739 heartbeater.cc:503] Master 127.1.127.254:35459 was elected leader, sending a full tablet report...
I20260807 15:06:00.622581 21880 heartbeater.cc:503] Master 127.1.127.254:35459 was elected leader, sending a full tablet report...
I20260807 15:06:00.623056 22044 log.cc:824] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:00.623996 22044 tablet_bootstrap.cc:492] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: No bootstrap required, opened a new log
I20260807 15:06:00.624087 22044 ts_tablet_manager.cc:1397] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:06:00.625756 22044 raft_consensus.cc:348] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.625936 22044 raft_consensus.cc:374] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:00.625965 22044 raft_consensus.cc:729] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe67dfbc24c14092a0b83ddb5d4171aa, State: Initialized, Role: FOLLOWER
I20260807 15:06:00.626106 22044 consensus_queue.cc:260] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.626467 22044 ts_tablet_manager.cc:1428] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:06:00.626547 22023 heartbeater.cc:503] Master 127.1.127.254:35459 was elected leader, sending a full tablet report...
W20260807 15:06:00.639885 21881 tablet.cc:2369] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:00.653388 22046 raft_consensus.cc:482] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:00.653527 22046 raft_consensus.cc:504] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.654083 22046 leader_election.cc:288] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe67dfbc24c14092a0b83ddb5d4171aa (127.1.127.195:39575), 9ab56d7a256c41258b15d9e3e739ae20 (127.1.127.194:35741)
I20260807 15:06:00.658679 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.654467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:43191 (local address 127.1.127.195:39575)
0807 15:06:00.654517 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.654519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.654794 (+ 275us) server_negotiation.cc:407] Connection header received
0807 15:06:00.654963 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.654964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.654972 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.655046 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:00.655246 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.656022 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.657577 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.658185 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.658279 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.658290 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.658521 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.658605 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:00.658722 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.654292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:35741 (local address 127.1.127.193:41185)
0807 15:06:00.654346 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.654360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.654410 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.654752 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.654753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.654756 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:00.654855 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.654857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.655966 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.655967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.657411 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.657415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.658368 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.658369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.658455 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.658458 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.658560 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.658598 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260807 15:06:00.658766 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.654454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:41185 (local address 127.1.127.194:35741)
0807 15:06:00.654566 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.654569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.654579 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:00.654614 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.654615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.654632 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.654736 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:00.654939 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.655849 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.657583 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.658278 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.658350 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.658358 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.658595 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.658645 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:06:00.658738 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.654296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:39575 (local address 127.1.127.193:43191)
0807 15:06:00.654663 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.654714 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:00.654800 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:00.655065 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:00.655065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:00.655067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:00.655171 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.655173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.656128 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.656129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.657418 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:00.657420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.658296 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.658297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:00.658413 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.658419 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:06:00.658495 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:06:00.658556 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":230,"thread_start_us":119,"threads_started":1}
I20260807 15:06:00.659188 21832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365d846e967c491e88c2b3bf4557a5e6" candidate_uuid: "7e0ae90692cf4421aac7af98529f1fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab56d7a256c41258b15d9e3e739ae20" is_pre_election: true
I20260807 15:06:00.659435 21832 raft_consensus.cc:2454] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e0ae90692cf4421aac7af98529f1fc8 in term 0.
I20260807 15:06:00.659742 21622 leader_election.cc:302] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e0ae90692cf4421aac7af98529f1fc8, 9ab56d7a256c41258b15d9e3e739ae20; no voters:
I20260807 15:06:00.659822 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365d846e967c491e88c2b3bf4557a5e6" candidate_uuid: "7e0ae90692cf4421aac7af98529f1fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" is_pre_election: true
I20260807 15:06:00.659951 22046 raft_consensus.cc:2781] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:00.659996 21974 raft_consensus.cc:2454] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e0ae90692cf4421aac7af98529f1fc8 in term 0.
I20260807 15:06:00.660058 22046 raft_consensus.cc:482] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:00.660101 22046 raft_consensus.cc:3037] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:00.660938 22046 raft_consensus.cc:504] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.661113 22046 leader_election.cc:288] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [CANDIDATE]: Term 1 election: Requested vote from peers fe67dfbc24c14092a0b83ddb5d4171aa (127.1.127.195:39575), 9ab56d7a256c41258b15d9e3e739ae20 (127.1.127.194:35741)
I20260807 15:06:00.661298 21832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365d846e967c491e88c2b3bf4557a5e6" candidate_uuid: "7e0ae90692cf4421aac7af98529f1fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab56d7a256c41258b15d9e3e739ae20"
I20260807 15:06:00.661309 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365d846e967c491e88c2b3bf4557a5e6" candidate_uuid: "7e0ae90692cf4421aac7af98529f1fc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa"
I20260807 15:06:00.661422 21974 raft_consensus.cc:3037] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:00.661427 21832 raft_consensus.cc:3037] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:00.662259 21974 raft_consensus.cc:2454] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e0ae90692cf4421aac7af98529f1fc8 in term 1.
I20260807 15:06:00.662422 21832 raft_consensus.cc:2454] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e0ae90692cf4421aac7af98529f1fc8 in term 1.
I20260807 15:06:00.662518 21624 leader_election.cc:302] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e0ae90692cf4421aac7af98529f1fc8, fe67dfbc24c14092a0b83ddb5d4171aa; no voters:
I20260807 15:06:00.662622 22046 raft_consensus.cc:2781] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:00.662875 22046 raft_consensus.cc:686] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 1 LEADER]: Becoming Leader. State: Replica: 7e0ae90692cf4421aac7af98529f1fc8, State: Running, Role: LEADER
I20260807 15:06:00.662992 22046 consensus_queue.cc:237] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } }
I20260807 15:06:00.664115 21539 catalog_manager.cc:5495] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e0ae90692cf4421aac7af98529f1fc8 (127.1.127.193). New cstate: current_term: 1 leader_uuid: "7e0ae90692cf4421aac7af98529f1fc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e0ae90692cf4421aac7af98529f1fc8" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 43493 } health_report { overall_health: HEALTHY } } }
I20260807 15:06:00.673301 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.670369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.1.127.193:43493)
0807 15:06:00.670438 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.670440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.670476 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:06:00.670562 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.670563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.670581 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.670657 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:00.670922 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.671459 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.672335 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.672576 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.672642 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.672835 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:00.672946 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:00.673072 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.673158 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.673180 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 15:06:00.675135 21740 tablet.cc:2369] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:00.675576 21832 raft_consensus.cc:1261] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 1 FOLLOWER]: Refusing update from remote peer 7e0ae90692cf4421aac7af98529f1fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:00.675588 21974 raft_consensus.cc:1261] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 1 FOLLOWER]: Refusing update from remote peer 7e0ae90692cf4421aac7af98529f1fc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:00.676024 22050 consensus_queue.cc:1035] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ab56d7a256c41258b15d9e3e739ae20" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 35741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:00.676193 22046 consensus_queue.cc:1035] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe67dfbc24c14092a0b83ddb5d4171aa" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:00.680495 22055 mvcc.cc:205] Tried to move back new op lower bound from 7315927698123280384 to 7315927698076516352. Current Snapshot: MvccSnapshot[applied={T|T < 7315927698123280384 or (T in {7315927698123280384})}]
I20260807 15:06:00.682054 21540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:00.682190 21540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:00.683116 21540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ce07a48502744dc89743b3a5ff21bf0: Sending DeleteTablet for 3 replicas of tablet 365d846e967c491e88c2b3bf4557a5e6
I20260807 15:06:00.683578 21672 tablet_service.cc:1514] Processing DeleteTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:00 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:38348
I20260807 15:06:00.683588 21951 tablet_service.cc:1514] Processing DeleteTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:00 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:51400
I20260807 15:06:00.683624 21810 tablet_service.cc:1514] Processing DeleteTablet for tablet 365d846e967c491e88c2b3bf4557a5e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:00 UTC) from {username='kudu', principal='kudu/127.1.127.254@KRBTEST.COM'} at 127.0.0.1:54738
I20260807 15:06:00.683852 22063 tablet_replica.cc:330] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8: stopping tablet replica
I20260807 15:06:00.683979 22063 raft_consensus.cc:2229] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:00.684095 22063 raft_consensus.cc:2258] T 365d846e967c491e88c2b3bf4557a5e6 P 7e0ae90692cf4421aac7af98529f1fc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:00.684303 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21604
I20260807 15:06:00.685851 22065 tablet_replica.cc:330] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: stopping tablet replica
I20260807 15:06:00.686038 22065 raft_consensus.cc:2229] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:00.686225 22065 raft_consensus.cc:2258] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:00.686466 22064 tablet_replica.cc:330] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: stopping tablet replica
I20260807 15:06:00.686620 22064 raft_consensus.cc:2229] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:00.686811 22064 raft_consensus.cc:2258] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:00.687465 22064 ts_tablet_manager.cc:1905] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:00.690433 22065 ts_tablet_manager.cc:1905] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:00.690722 22064 ts_tablet_manager.cc:1918] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:00.690842 22064 log.cc:1196] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/365d846e967c491e88c2b3bf4557a5e6
I20260807 15:06:00.691681 22064 ts_tablet_manager.cc:1939] T 365d846e967c491e88c2b3bf4557a5e6 P fe67dfbc24c14092a0b83ddb5d4171aa: Deleting consensus metadata
I20260807 15:06:00.692652 21527 catalog_manager.cc:4841] TS fe67dfbc24c14092a0b83ddb5d4171aa (127.1.127.195:39575): tablet 365d846e967c491e88c2b3bf4557a5e6 (table test-table [id=dbb70d68068540739c13a614ec8d49b1]) successfully deleted
I20260807 15:06:00.693262 22065 ts_tablet_manager.cc:1918] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:00.693372 22065 log.cc:1196] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/365d846e967c491e88c2b3bf4557a5e6
I20260807 15:06:00.693707 22065 ts_tablet_manager.cc:1939] T 365d846e967c491e88c2b3bf4557a5e6 P 9ab56d7a256c41258b15d9e3e739ae20: Deleting consensus metadata
I20260807 15:06:00.694258 21525 catalog_manager.cc:4841] TS 9ab56d7a256c41258b15d9e3e739ae20 (127.1.127.194:35741): tablet 365d846e967c491e88c2b3bf4557a5e6 (table test-table [id=dbb70d68068540739c13a614ec8d49b1]) successfully deleted
W20260807 15:06:00.696074 21525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:43493 (error 108)
I20260807 15:06:00.696265 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21745
I20260807 15:06:00.698060 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.697796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:43493 (local address 127.0.0.1:38374)
0807 15:06:00.697957 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:00.698011 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:43493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260807 15:06:00.698271 21525 catalog_manager.cc:4584] TS 7e0ae90692cf4421aac7af98529f1fc8 (127.1.127.193:43493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 365d846e967c491e88c2b3bf4557a5e6: Network error: Client connection negotiation failed: client connection to 127.1.127.193:43493: connect: Connection refused (error 111)
I20260807 15:06:00.703161 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21885
I20260807 15:06:00.709510 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 21506
2026-08-07T15:06:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T15:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:00Z Disabled control of system clock
I20260807 15:06:00.741837 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46883
--webserver_interface=127.1.127.254
--webserver_port=0
--builtin_ntp_servers=127.1.127.212:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:06:00.840602 22070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:00.840873 22070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:00.840898 22070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:00.842381 22070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:00.842443 22070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:00.842458 22070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:00.842471 22070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:00.842484 22070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:00.844069 22070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:46883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:00.844364 22070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:00.844699 22070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:00.850028 22078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.850039 22076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:00.850112 22075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:00.850400 22070 server_base.cc:1034] running on GCE node
I20260807 15:06:00.850807 22070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:00.851141 22070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:00.852339 22070 hybrid_clock.cc:648] HybridClock initialized: now 1786115160852321 us; error 35 us; skew 500 ppm
I20260807 15:06:00.854034 22070 webserver.cc:460] Webserver started at http://127.1.127.254:45101/ using document root <none> and password file <none>
I20260807 15:06:00.854357 22070 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:00.854413 22070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:00.854586 22070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:00.855763 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "9d7f2177f61946789b0e8f8c8cba79b8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9b8a2cee17b749df36c7f176c97a70be"
server_key_iv: "0e9c92110d61337d14a4cad06adee105"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.856146 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "9d7f2177f61946789b0e8f8c8cba79b8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9b8a2cee17b749df36c7f176c97a70be"
server_key_iv: "0e9c92110d61337d14a4cad06adee105"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.857745 22070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:00.858772 22084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:00.859027 22070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:00.859164 22070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "9d7f2177f61946789b0e8f8c8cba79b8"
format_stamp: "Formatted at 2026-08-07 15:06:00 on dist-test-slave-8tdl"
server_key: "9b8a2cee17b749df36c7f176c97a70be"
server_key_iv: "0e9c92110d61337d14a4cad06adee105"
server_key_version: "encryptionkey@0"
I20260807 15:06:00.859282 22070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:00.869339 22070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:00.869735 22070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:00.869913 22070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:00.875241 22070 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:46883
I20260807 15:06:00.875284 22140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:46883 every 8 connection(s)
I20260807 15:06:00.875710 22070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:00.876997 22141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:00.878152 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22070
I20260807 15:06:00.878250 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:00.878461 1535 external_mini_cluster.cc:1428] Setting key b1a006c43d9d63f51ceddb5ce3505a94
I20260807 15:06:00.881458 22141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: Bootstrap starting.
I20260807 15:06:00.882226 22141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:00.882571 22141 log.cc:824] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:00.883473 22141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: No bootstrap required, opened a new log
I20260807 15:06:00.885448 22141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } }
I20260807 15:06:00.885574 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:00.879090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.1.127.254:46883)
0807 15:06:00.879319 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:00.879328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:00.879362 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:06:00.879944 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:00.879953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:00.880050 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:00.880202 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:00.880531 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.881995 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.883197 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:00.883721 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:00.883920 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:00.884642 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:00.884647 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:00.884648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:00.884829 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:00.884865 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:00.884877 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:00.884959 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:00.885037 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:06:00.885125 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260807 15:06:00.885604 22141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:00.885695 22141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7f2177f61946789b0e8f8c8cba79b8, State: Initialized, Role: FOLLOWER
I20260807 15:06:00.885846 22141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } }
I20260807 15:06:00.885933 22141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:00.886003 22141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:00.886049 22141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:00.886963 22141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } }
I20260807 15:06:00.887449 22141 leader_election.cc:302] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d7f2177f61946789b0e8f8c8cba79b8; no voters:
I20260807 15:06:00.887709 22141 leader_election.cc:288] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:00.887816 22146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:00.888008 22146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [term 1 LEADER]: Becoming Leader. State: Replica: 9d7f2177f61946789b0e8f8c8cba79b8, State: Running, Role: LEADER
I20260807 15:06:00.888084 22141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:00.888116 22146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } }
I20260807 15:06:00.888546 22147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } } }
I20260807 15:06:00.888689 22147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:00.888545 22148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d7f2177f61946789b0e8f8c8cba79b8. Latest consensus state: current_term: 1 leader_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f2177f61946789b0e8f8c8cba79b8" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46883 } } }
I20260807 15:06:00.888813 22148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:00.888962 22156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:00.889569 22156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:00.891517 22156 catalog_manager.cc:1347] Generated new cluster ID: 5ca5a193149844089fdcce4e159cf5d1
I20260807 15:06:00.891582 22156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:00.918350 22156 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:00.919165 22156 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:00.926054 22156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: Generated new TSK 0
I20260807 15:06:00.926285 22156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:00.936476 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.193:0
--local_ip_for_outbound_sockets=127.1.127.193
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--builtin_ntp_servers=127.1.127.212:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:06:01.028505 22166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:01.028779 22166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:01.028823 22166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:01.030202 22166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:01.030603 22166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:01.030748 22166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.193
I20260807 15:06:01.033142 22166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.127.193
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.193
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:01.033491 22166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:01.033851 22166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:01.039882 22171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.039908 22172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.040225 22174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:01.040364 22166 server_base.cc:1034] running on GCE node
I20260807 15:06:01.040738 22166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:01.041043 22166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:01.042227 22166 hybrid_clock.cc:648] HybridClock initialized: now 1786115161042212 us; error 32 us; skew 500 ppm
I20260807 15:06:01.043794 22166 webserver.cc:460] Webserver started at http://127.1.127.193:36687/ using document root <none> and password file <none>
I20260807 15:06:01.044134 22166 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:01.044222 22166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:01.044437 22166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:01.045831 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data/instance:
uuid: "83a48a5a5a59443a927c6474daaf8aca"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "805d658fa5d21627ad6717479a4e9f39"
server_key_iv: "f10fa183786f08807bc101dc889299b8"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.046262 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance:
uuid: "83a48a5a5a59443a927c6474daaf8aca"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "805d658fa5d21627ad6717479a4e9f39"
server_key_iv: "f10fa183786f08807bc101dc889299b8"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.048154 22166 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:06:01.049387 22181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.049741 22166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:01.049893 22166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal
uuid: "83a48a5a5a59443a927c6474daaf8aca"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "805d658fa5d21627ad6717479a4e9f39"
server_key_iv: "f10fa183786f08807bc101dc889299b8"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.050014 22166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:01.090209 22166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:01.090807 22166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:01.091013 22166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:01.091300 22166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:01.091688 22166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:01.091749 22166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.091786 22166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:01.091836 22166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.098439 22166 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.193:42699
I20260807 15:06:01.098489 22301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.193:42699 every 8 connection(s)
I20260807 15:06:01.098933 22166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:01.104040 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22166
I20260807 15:06:01.104163 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:01.104387 1535 external_mini_cluster.cc:1428] Setting key aa774fa58ff83c0d874d3d6db064b513
I20260807 15:06:01.105494 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.194:0
--local_ip_for_outbound_sockets=127.1.127.194
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--builtin_ntp_servers=127.1.127.212:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:06:01.106294 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.100135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.193:56095 (local address 127.1.127.254:46883)
0807 15:06:01.100235 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.100238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.100439 (+ 201us) server_negotiation.cc:407] Connection header received
0807 15:06:01.101119 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.101120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.101132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.101194 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:01.102055 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.102601 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.104242 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.104630 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.104736 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.105944 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:01.105946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:01.105946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:01.105951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:01.105968 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:01.105973 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:01.106048 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.106162 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.106208 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:06:01.106614 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.100067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46883 (local address 127.1.127.193:56095)
0807 15:06:01.100303 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.100343 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.100880 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.101345 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.101349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.101358 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:01.101902 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.101907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.102752 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.102753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.103964 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.103995 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.104771 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.104773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.104965 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.105670 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:06:01.105671 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:01.105717 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:06:01.105758 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:01.106052 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:01.106061 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:01.106067 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.106162 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.106294 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":120,"threads_started":1}
I20260807 15:06:01.107236 22302 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46883
I20260807 15:06:01.107417 22302 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:01.107631 22302 heartbeater.cc:511] Master 127.1.127.254:46883 requested a full tablet report, sending...
I20260807 15:06:01.108243 22101 ts_manager.cc:194] Registered new tserver with Master: 83a48a5a5a59443a927c6474daaf8aca (127.1.127.193:42699)
I20260807 15:06:01.108913 22101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.193:56095
W20260807 15:06:01.197247 22305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:01.197528 22305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:01.197566 22305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:01.198875 22305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:01.198940 22305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:01.198987 22305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.194
I20260807 15:06:01.200464 22305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.127.194
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.194
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:01.200779 22305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:01.201139 22305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:01.206921 22305 server_base.cc:1034] running on GCE node
W20260807 15:06:01.206887 22313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.206887 22311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.206890 22310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:01.207422 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:01.207688 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:01.208902 22305 hybrid_clock.cc:648] HybridClock initialized: now 1786115161208878 us; error 41 us; skew 500 ppm
I20260807 15:06:01.210522 22305 webserver.cc:460] Webserver started at http://127.1.127.194:33419/ using document root <none> and password file <none>
I20260807 15:06:01.210933 22305 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:01.210995 22305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:01.211161 22305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:01.212555 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data/instance:
uuid: "51b8b7960a3f4f538a7eb8b7fa464490"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "798f4c89aacb0cdba8618819b4035a50"
server_key_iv: "b5bb688b3b42f68c6d062c584265215f"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.212975 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance:
uuid: "51b8b7960a3f4f538a7eb8b7fa464490"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "798f4c89aacb0cdba8618819b4035a50"
server_key_iv: "b5bb688b3b42f68c6d062c584265215f"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.214620 22305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:06:01.215701 22319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.215926 22305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:01.216037 22305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal
uuid: "51b8b7960a3f4f538a7eb8b7fa464490"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "798f4c89aacb0cdba8618819b4035a50"
server_key_iv: "b5bb688b3b42f68c6d062c584265215f"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.216133 22305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:01.231802 22305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:01.232198 22305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:01.232370 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:01.232645 22305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:01.233033 22305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:01.233093 22305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.233130 22305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:01.233158 22305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.239549 22305 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.194:39401
I20260807 15:06:01.239660 22437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.194:39401 every 8 connection(s)
I20260807 15:06:01.240000 22305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:01.242669 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22305
I20260807 15:06:01.242856 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:01.243115 1535 external_mini_cluster.cc:1428] Setting key 53a566a380e126f1824ba2339e29707a
I20260807 15:06:01.244354 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.127.195:0
--local_ip_for_outbound_sockets=127.1.127.195
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--builtin_ntp_servers=127.1.127.212:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:06:01.246819 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.240850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57669 (local address 127.1.127.254:46883)
0807 15:06:01.240985 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.240987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.241191 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:06:01.241803 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.241804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.241818 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.241900 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:01.242625 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.243349 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.244959 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.245161 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.245237 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.246501 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:01.246503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:01.246503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:01.246509 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:01.246526 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:01.246530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:01.246587 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.246692 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.246734 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 15:06:01.247102 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.240805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46883 (local address 127.1.127.194:57669)
0807 15:06:01.241053 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.241096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.241673 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.241978 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.241981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.241989 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:01.242524 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.242528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.243513 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.243514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.244831 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.244835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.245293 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.245294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.245521 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.246331 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:06:01.246332 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:01.246366 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:06:01.246400 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:01.246597 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:01.246602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:01.246606 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.246707 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.246831 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260807 15:06:01.247833 22438 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46883
I20260807 15:06:01.247953 22438 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:01.248131 22438 heartbeater.cc:511] Master 127.1.127.254:46883 requested a full tablet report, sending...
I20260807 15:06:01.248618 22101 ts_manager.cc:194] Registered new tserver with Master: 51b8b7960a3f4f538a7eb8b7fa464490 (127.1.127.194:39401)
I20260807 15:06:01.249063 22101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.194:57669
W20260807 15:06:01.336200 22441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:01.336485 22441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:01.336527 22441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:01.337826 22441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:01.337908 22441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:01.337952 22441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.127.195
I20260807 15:06:01.339564 22441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.127.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.127.195
--webserver_port=0
--tserver_master_addrs=127.1.127.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.127.195
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:01.339883 22441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:01.340231 22441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:01.346181 22449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.346223 22447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:01.346189 22446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:01.346608 22441 server_base.cc:1034] running on GCE node
I20260807 15:06:01.346902 22441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:01.347199 22441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:01.348362 22441 hybrid_clock.cc:648] HybridClock initialized: now 1786115161348343 us; error 42 us; skew 500 ppm
I20260807 15:06:01.350178 22441 webserver.cc:460] Webserver started at http://127.1.127.195:45909/ using document root <none> and password file <none>
I20260807 15:06:01.350519 22441 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:01.350577 22441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:01.350790 22441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:01.352049 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data/instance:
uuid: "870785be476640e18885fbfc1fb35447"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "8c0699569cc6719af991d3dd59fba684"
server_key_iv: "c22f93d2e0fcab2976557906d9e0f457"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.352450 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance:
uuid: "870785be476640e18885fbfc1fb35447"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "8c0699569cc6719af991d3dd59fba684"
server_key_iv: "c22f93d2e0fcab2976557906d9e0f457"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.354465 22441 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260807 15:06:01.356478 22455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.357007 22441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 15:06:01.357146 22441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal
uuid: "870785be476640e18885fbfc1fb35447"
format_stamp: "Formatted at 2026-08-07 15:06:01 on dist-test-slave-8tdl"
server_key: "8c0699569cc6719af991d3dd59fba684"
server_key_iv: "c22f93d2e0fcab2976557906d9e0f457"
server_key_version: "encryptionkey@0"
I20260807 15:06:01.357254 22441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:01.376580 22441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:01.376976 22441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:01.377151 22441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:01.377440 22441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:01.377875 22441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:01.377986 22441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.378039 22441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:01.378170 22441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:01.385572 22441 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.195:41541
I20260807 15:06:01.385617 22577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.195:41541 every 8 connection(s)
I20260807 15:06:01.386070 22441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:01.392228 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.386960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.195:41347 (local address 127.1.127.254:46883)
0807 15:06:01.387022 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.387025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.387317 (+ 292us) server_negotiation.cc:407] Connection header received
0807 15:06:01.387881 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.387882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.387895 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.387954 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:01.388754 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.389358 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.390550 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.390840 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.390922 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.391892 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:01.391893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:01.391894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:01.391905 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:01.391921 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:01.391926 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:01.392003 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.392102 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.392134 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:06:01.392495 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.386966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:46883 (local address 127.1.127.195:41347)
0807 15:06:01.387169 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.387207 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.387714 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.388033 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.388037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.388048 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:01.388657 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.388662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.389475 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.389477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.390449 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.390453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.390936 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.390937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.391074 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.391716 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:06:01.391717 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:01.391749 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:06:01.391778 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:01.392008 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:01.392015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:01.392019 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.392103 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.392226 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260807 15:06:01.393137 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22441
I20260807 15:06:01.393236 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:01.393365 22578 heartbeater.cc:347] Connected to a master server at 127.1.127.254:46883
I20260807 15:06:01.393514 22578 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:01.393587 1535 external_mini_cluster.cc:1428] Setting key a62cb37cb6ec5bb0d3bbf9f773d18cae
I20260807 15:06:01.393769 22578 heartbeater.cc:511] Master 127.1.127.254:46883 requested a full tablet report, sending...
I20260807 15:06:01.394258 22101 ts_manager.cc:194] Registered new tserver with Master: 870785be476640e18885fbfc1fb35447 (127.1.127.195:41541)
I20260807 15:06:01.394856 1535 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:01.394939 22101 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.127.195:41347
I20260807 15:06:01.399981 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.396518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.1.127.254:46883)
0807 15:06:01.396591 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.396594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.396736 (+ 142us) server_negotiation.cc:407] Connection header received
0807 15:06:01.396832 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.396833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.396845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.396903 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:01.397149 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.397708 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.398541 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.398859 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.398947 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.399712 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:01.399713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:01.399714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:01.399719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:01.399735 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:01.399740 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:01.399793 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.399861 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.399889 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:06:01.401909 22101 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:01.403963 22101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:01.411139 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.407209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:42699 (local address 127.0.0.1:47230)
0807 15:06:01.407593 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.407603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.407653 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.407889 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.407890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.407892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:01.407960 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.407962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.408843 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.408844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.409880 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.409882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.410797 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.410798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.410888 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.410912 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.410989 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.411045 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":59,"threads_started":1}
I20260807 15:06:01.411144 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.407328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.1.127.193:42699)
0807 15:06:01.407518 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.407523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.407712 (+ 189us) server_negotiation.cc:407] Connection header received
0807 15:06:01.407760 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.407762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.407778 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.407892 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:01.408019 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.408750 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.409993 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.410722 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.410800 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.410907 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.411001 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.411046 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260807 15:06:01.411684 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.406930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.194:39401 (local address 127.0.0.1:47838)
0807 15:06:01.407111 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.407163 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.407243 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.407738 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.407742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.407748 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:01.407844 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.407847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.409567 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.409569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.410382 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.410384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.411437 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.411438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.411506 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.411510 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.411554 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.411602 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260807 15:06:01.411697 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.407310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.1.127.194:39401)
0807 15:06:01.407519 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.407524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.407536 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:01.407584 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.407587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.407605 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.408070 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:01.408137 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.409402 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.410492 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.411333 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.411434 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.411525 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.411564 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.411602 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260807 15:06:01.412412 22232 tablet_service.cc:1467] Processing CreateTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd (DEFAULT_TABLE table=test-table [id=c1dc3ebffa1c4602b574573c8cd8801c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:01.412672 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.406989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41541 (local address 127.0.0.1:33766)
0807 15:06:01.407319 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.407329 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.407382 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.407738 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.407742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.407744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:01.407944 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.407945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.410467 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.410467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.411467 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.411470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.412427 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.412428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.412494 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.412499 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.412549 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.412599 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":145,"thread_start_us":54,"threads_started":1}
I20260807 15:06:01.412701 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.407082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.1.127.195:41541)
0807 15:06:01.407547 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.407552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.407562 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:01.407602 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.407604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.407619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.407785 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:01.408033 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.409397 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.411570 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.412297 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.412422 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.412513 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.412578 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.412619 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":112,"threads_started":1}
I20260807 15:06:01.412871 22232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17d6e4b9f15481d98a4fcc04ab858cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:01.413307 22508 tablet_service.cc:1467] Processing CreateTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd (DEFAULT_TABLE table=test-table [id=c1dc3ebffa1c4602b574573c8cd8801c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:01.413640 22508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17d6e4b9f15481d98a4fcc04ab858cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:01.415926 22593 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: Bootstrap starting.
I20260807 15:06:01.416365 22594 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Bootstrap starting.
I20260807 15:06:01.416460 22368 tablet_service.cc:1467] Processing CreateTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd (DEFAULT_TABLE table=test-table [id=c1dc3ebffa1c4602b574573c8cd8801c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:01.416836 22593 tablet_bootstrap.cc:654] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:01.416841 22368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17d6e4b9f15481d98a4fcc04ab858cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:01.417037 22594 tablet_bootstrap.cc:654] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:01.417197 22593 log.cc:824] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:01.417392 22594 log.cc:824] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:01.418028 22593 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: No bootstrap required, opened a new log
I20260807 15:06:01.418118 22593 ts_tablet_manager.cc:1397] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:01.418170 22594 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: No bootstrap required, opened a new log
I20260807 15:06:01.418242 22594 ts_tablet_manager.cc:1397] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:06:01.420303 22593 raft_consensus.cc:348] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.420539 22593 raft_consensus.cc:374] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:01.420586 22593 raft_consensus.cc:729] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a48a5a5a59443a927c6474daaf8aca, State: Initialized, Role: FOLLOWER
I20260807 15:06:01.420723 22593 consensus_queue.cc:260] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.421092 22593 ts_tablet_manager.cc:1428] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:06:01.421099 22597 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Bootstrap starting.
I20260807 15:06:01.421238 22302 heartbeater.cc:503] Master 127.1.127.254:46883 was elected leader, sending a full tablet report...
I20260807 15:06:01.421124 22594 raft_consensus.cc:348] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.421306 22594 raft_consensus.cc:374] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:01.421330 22594 raft_consensus.cc:729] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870785be476640e18885fbfc1fb35447, State: Initialized, Role: FOLLOWER
I20260807 15:06:01.421417 22594 consensus_queue.cc:260] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.421686 22594 ts_tablet_manager.cc:1428] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 15:06:01.421710 22578 heartbeater.cc:503] Master 127.1.127.254:46883 was elected leader, sending a full tablet report...
I20260807 15:06:01.422326 22597 tablet_bootstrap.cc:654] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:01.422775 22597 log.cc:824] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:01.423583 22597 tablet_bootstrap.cc:492] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: No bootstrap required, opened a new log
I20260807 15:06:01.423652 22597 ts_tablet_manager.cc:1397] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 15:06:01.426010 22597 raft_consensus.cc:348] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.426218 22597 raft_consensus.cc:374] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:01.426275 22597 raft_consensus.cc:729] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b8b7960a3f4f538a7eb8b7fa464490, State: Initialized, Role: FOLLOWER
I20260807 15:06:01.426424 22597 consensus_queue.cc:260] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.426818 22597 ts_tablet_manager.cc:1428] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:06:01.426860 22438 heartbeater.cc:503] Master 127.1.127.254:46883 was elected leader, sending a full tablet report...
W20260807 15:06:01.490619 22439 tablet.cc:2369] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:01.526201 22601 raft_consensus.cc:482] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:01.526338 22601 raft_consensus.cc:504] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.526774 22601 leader_election.cc:288] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 870785be476640e18885fbfc1fb35447 (127.1.127.195:41541), 83a48a5a5a59443a927c6474daaf8aca (127.1.127.193:42699)
I20260807 15:06:01.531543 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.527049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.195:41541 (local address 127.1.127.194:53755)
0807 15:06:01.527106 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.527121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.527185 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.527651 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.527653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.527655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:01.527775 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.527778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.528939 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.528941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.530312 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.530317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.531206 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.531207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.531300 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.531304 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.531363 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.531428 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":80}
I20260807 15:06:01.531549 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.527209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:42699 (local address 127.1.127.194:57861)
0807 15:06:01.527465 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.527474 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:01.527530 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:01.527716 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:01.527717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:01.527719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:01.527795 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.527797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.528939 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.528941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.530312 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:01.530317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.531206 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.531207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:01.531306 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.531309 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:01.531366 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:06:01.531421 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":22,"thread_start_us":125,"threads_started":1}
I20260807 15:06:01.531724 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.527304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:53755 (local address 127.1.127.195:41541)
0807 15:06:01.527462 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.527465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.527485 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:06:01.527534 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.527535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.527547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.527617 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:01.527870 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.528788 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.530443 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.531056 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.531141 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.531149 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.531598 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.531654 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:06:01.531852 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.527314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.127.194:57861 (local address 127.1.127.193:42699)
0807 15:06:01.527370 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.527372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.527580 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:06:01.527613 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.527614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.527626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.527681 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:01.527876 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.528795 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.530443 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.531067 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.531122 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.531133 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.531590 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.531725 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:06:01.532490 22529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17d6e4b9f15481d98a4fcc04ab858cd" candidate_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870785be476640e18885fbfc1fb35447" is_pre_election: true
I20260807 15:06:01.532490 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17d6e4b9f15481d98a4fcc04ab858cd" candidate_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a48a5a5a59443a927c6474daaf8aca" is_pre_election: true
I20260807 15:06:01.532680 22253 raft_consensus.cc:2454] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b8b7960a3f4f538a7eb8b7fa464490 in term 0.
I20260807 15:06:01.532680 22529 raft_consensus.cc:2454] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b8b7960a3f4f538a7eb8b7fa464490 in term 0.
I20260807 15:06:01.532948 22323 leader_election.cc:302] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b8b7960a3f4f538a7eb8b7fa464490, 83a48a5a5a59443a927c6474daaf8aca; no voters:
I20260807 15:06:01.533097 22601 raft_consensus.cc:2781] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:01.533161 22601 raft_consensus.cc:482] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:01.533200 22601 raft_consensus.cc:3037] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:01.534111 22601 raft_consensus.cc:504] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.534439 22529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17d6e4b9f15481d98a4fcc04ab858cd" candidate_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870785be476640e18885fbfc1fb35447"
I20260807 15:06:01.534549 22529 raft_consensus.cc:3037] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:01.535431 22529 raft_consensus.cc:2454] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b8b7960a3f4f538a7eb8b7fa464490 in term 1.
I20260807 15:06:01.535573 22601 leader_election.cc:288] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [CANDIDATE]: Term 1 election: Requested vote from peers 870785be476640e18885fbfc1fb35447 (127.1.127.195:41541), 83a48a5a5a59443a927c6474daaf8aca (127.1.127.193:42699)
I20260807 15:06:01.535656 22323 leader_election.cc:302] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b8b7960a3f4f538a7eb8b7fa464490, 870785be476640e18885fbfc1fb35447; no voters:
I20260807 15:06:01.535750 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17d6e4b9f15481d98a4fcc04ab858cd" candidate_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a48a5a5a59443a927c6474daaf8aca"
I20260807 15:06:01.535864 22253 raft_consensus.cc:3037] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:01.536689 22253 raft_consensus.cc:2454] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b8b7960a3f4f538a7eb8b7fa464490 in term 1.
I20260807 15:06:01.536904 22601 raft_consensus.cc:2781] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:01.537081 22601 raft_consensus.cc:686] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 1 LEADER]: Becoming Leader. State: Replica: 51b8b7960a3f4f538a7eb8b7fa464490, State: Running, Role: LEADER
I20260807 15:06:01.537204 22601 consensus_queue.cc:237] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } }
I20260807 15:06:01.538390 22101 catalog_manager.cc:5495] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b8b7960a3f4f538a7eb8b7fa464490 (127.1.127.194). New cstate: current_term: 1 leader_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b8b7960a3f4f538a7eb8b7fa464490" member_type: VOTER last_known_addr { host: "127.1.127.194" port: 39401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:01.568477 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.565803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.1.127.194:39401)
0807 15:06:01.565884 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:01.565886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:01.565895 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:01.565943 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:01.565944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:01.565964 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:01.566046 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:01.566201 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.566777 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.567592 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:01.567800 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:01.567884 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:01.568057 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:01.568204 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:01.568312 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:01.568332 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 15:06:01.568354 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:06:01.570601 22253 raft_consensus.cc:1261] T a17d6e4b9f15481d98a4fcc04ab858cd P 83a48a5a5a59443a927c6474daaf8aca [term 1 FOLLOWER]: Refusing update from remote peer 51b8b7960a3f4f538a7eb8b7fa464490: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:01.570623 22529 raft_consensus.cc:1261] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 1 FOLLOWER]: Refusing update from remote peer 51b8b7960a3f4f538a7eb8b7fa464490: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:01.571133 22603 consensus_queue.cc:1035] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [LEADER]: Connected to new peer: Peer: permanent_uuid: "870785be476640e18885fbfc1fb35447" member_type: VOTER last_known_addr { host: "127.1.127.195" port: 41541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:01.571288 22601 consensus_queue.cc:1035] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83a48a5a5a59443a927c6474daaf8aca" member_type: VOTER last_known_addr { host: "127.1.127.193" port: 42699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:01.574892 22610 mvcc.cc:205] Tried to move back new op lower bound from 7315927701789536256 to 7315927701657034752. Current Snapshot: MvccSnapshot[applied={T|T < 7315927701789536256}]
I20260807 15:06:01.576099 22608 mvcc.cc:205] Tried to move back new op lower bound from 7315927701789536256 to 7315927701657034752. Current Snapshot: MvccSnapshot[applied={T|T < 7315927701789536256}]
I20260807 15:06:01.577363 22101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:01.577509 22101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:01.578593 22101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d7f2177f61946789b0e8f8c8cba79b8: Sending DeleteTablet for 3 replicas of tablet a17d6e4b9f15481d98a4fcc04ab858cd
I20260807 15:06:01.579056 22368 tablet_service.cc:1514] Processing DeleteTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:01 UTC) from {username='slave'} at 127.0.0.1:47838
I20260807 15:06:01.579071 22508 tablet_service.cc:1514] Processing DeleteTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:01 UTC) from {username='slave'} at 127.0.0.1:33766
I20260807 15:06:01.579106 22232 tablet_service.cc:1514] Processing DeleteTablet for tablet a17d6e4b9f15481d98a4fcc04ab858cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:01 UTC) from {username='slave'} at 127.0.0.1:47230
I20260807 15:06:01.579341 22619 tablet_replica.cc:330] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: stopping tablet replica
I20260807 15:06:01.579356 22618 tablet_replica.cc:330] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: stopping tablet replica
I20260807 15:06:01.579443 22619 raft_consensus.cc:2229] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:01.579443 22618 raft_consensus.cc:2229] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:01.579538 22618 raft_consensus.cc:2258] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:01.579604 22619 raft_consensus.cc:2258] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:01.579782 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22166
I20260807 15:06:01.580083 22618 ts_tablet_manager.cc:1905] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:01.581179 22619 ts_tablet_manager.cc:1905] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:01.582466 22618 ts_tablet_manager.cc:1918] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:01.582537 22618 log.cc:1196] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-1/wal/wals/a17d6e4b9f15481d98a4fcc04ab858cd
I20260807 15:06:01.582870 22618 ts_tablet_manager.cc:1939] T a17d6e4b9f15481d98a4fcc04ab858cd P 51b8b7960a3f4f538a7eb8b7fa464490: Deleting consensus metadata
I20260807 15:06:01.582984 22619 ts_tablet_manager.cc:1918] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:01.583052 22619 log.cc:1196] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Deleting WAL directory at /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786114902562993-1535-0/minicluster-data/ts-2/wal/wals/a17d6e4b9f15481d98a4fcc04ab858cd
I20260807 15:06:01.583312 22619 ts_tablet_manager.cc:1939] T a17d6e4b9f15481d98a4fcc04ab858cd P 870785be476640e18885fbfc1fb35447: Deleting consensus metadata
I20260807 15:06:01.583379 22087 catalog_manager.cc:4841] TS 51b8b7960a3f4f538a7eb8b7fa464490 (127.1.127.194:39401): tablet a17d6e4b9f15481d98a4fcc04ab858cd (table test-table [id=c1dc3ebffa1c4602b574573c8cd8801c]) successfully deleted
I20260807 15:06:01.583879 22088 catalog_manager.cc:4841] TS 870785be476640e18885fbfc1fb35447 (127.1.127.195:41541): tablet a17d6e4b9f15481d98a4fcc04ab858cd (table test-table [id=c1dc3ebffa1c4602b574573c8cd8801c]) successfully deleted
W20260807 15:06:01.590806 22088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.127.193:42699 (error 108)
I20260807 15:06:01.591018 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22305
I20260807 15:06:01.591832 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:01.591345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.193:42699 (local address 127.0.0.1:47236)
0807 15:06:01.591712 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:01.591771 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.193:42699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339}
W20260807 15:06:01.592974 22088 catalog_manager.cc:4584] TS 83a48a5a5a59443a927c6474daaf8aca (127.1.127.193:42699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a17d6e4b9f15481d98a4fcc04ab858cd: Network error: Client connection negotiation failed: client connection to 127.1.127.193:42699: connect: Connection refused (error 111)
I20260807 15:06:01.597836 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22441
I20260807 15:06:01.603751 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22070
2026-08-07T15:06:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (893 ms)
[----------] 6 tests from EncryptionPolicyTest (9335 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:01 dist-test-slave-8tdl krb5kdc[22624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:01 dist-test-slave-8tdl krb5kdc[22624](info): set up 2 sockets
Aug 07 15:06:01 dist-test-slave-8tdl krb5kdc[22624](info): commencing operation
krb5kdc: starting...
W20260807 15:06:02.723102 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:02 dist-test-slave-8tdl krb5kdc[22624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:02.874369 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:02.964180 22637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:02.964512 22637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:02.964579 22637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:02.966171 22637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:02.966274 22637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:02.966305 22637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:02.966341 22637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:02.966379 22637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:02.968406 22637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:02.968765 22637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:02.969058 22637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:02.975921 22642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:02.975948 22643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:02.976564 22637 server_base.cc:1034] running on GCE node
W20260807 15:06:02.975921 22645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:02.977047 22637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:02.977445 22637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:02.978623 22637 hybrid_clock.cc:648] HybridClock initialized: now 1786115162978584 us; error 60 us; skew 500 ppm
Aug 07 15:06:02 dist-test-slave-8tdl krb5kdc[22624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:02.988111 22637 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:02.988770 22637 webserver.cc:460] Webserver started at http://127.0.0.1:38875/ using document root <none> and password file <none>
I20260807 15:06:02.989024 22637 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:02.989073 22637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:02.989181 22637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:02.990443 22637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "4eb92e9162b347c8a590bdff6261e32d"
format_stamp: "Formatted at 2026-08-07 15:06:02 on dist-test-slave-8tdl"
server_key: "bda09eadf907a0a0c78fc9aa217482c9"
server_key_iv: "1667f1ab2f4f99c04b2b533543f10d08"
server_key_version: "encryptionkey@0"
I20260807 15:06:02.990906 22637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "4eb92e9162b347c8a590bdff6261e32d"
format_stamp: "Formatted at 2026-08-07 15:06:02 on dist-test-slave-8tdl"
server_key: "bda09eadf907a0a0c78fc9aa217482c9"
server_key_iv: "1667f1ab2f4f99c04b2b533543f10d08"
server_key_version: "encryptionkey@0"
I20260807 15:06:02.992578 22637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:02.993474 22652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:02.993716 22637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:02.993819 22637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "4eb92e9162b347c8a590bdff6261e32d"
format_stamp: "Formatted at 2026-08-07 15:06:02 on dist-test-slave-8tdl"
server_key: "bda09eadf907a0a0c78fc9aa217482c9"
server_key_iv: "1667f1ab2f4f99c04b2b533543f10d08"
server_key_version: "encryptionkey@0"
I20260807 15:06:02.993880 22637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:03.008009 22637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:03.011708 22637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:03.011849 22637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:03.018509 22637 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38067
I20260807 15:06:03.018628 22704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38067 every 8 connection(s)
I20260807 15:06:03.019013 22637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:03.020270 22705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:03.021376 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22637
I20260807 15:06:03.021561 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:03.021790 1535 external_mini_cluster.cc:1428] Setting key 978ab487d32d8a8aeda5e3800b5ea8e3
I20260807 15:06:03.023684 22705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: Bootstrap starting.
I20260807 15:06:03.024695 22705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:03.025096 22705 log.cc:824] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:03.026075 22705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: No bootstrap required, opened a new log
I20260807 15:06:03.028882 22705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } }
I20260807 15:06:03.029053 22705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:03.029079 22705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb92e9162b347c8a590bdff6261e32d, State: Initialized, Role: FOLLOWER
I20260807 15:06:03.029238 22705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } }
I20260807 15:06:03.029338 22705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:03.029412 22705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:03.029471 22705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:03.030265 22705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } }
I20260807 15:06:03.030428 22705 leader_election.cc:302] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eb92e9162b347c8a590bdff6261e32d; no voters:
Aug 07 15:06:03 dist-test-slave-8tdl krb5kdc[22624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:06:03.030715 22705 leader_election.cc:288] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:03.030761 22712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:03.030936 22712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [term 1 LEADER]: Becoming Leader. State: Replica: 4eb92e9162b347c8a590bdff6261e32d, State: Running, Role: LEADER
I20260807 15:06:03.031042 22705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:03.031033 22712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } }
I20260807 15:06:03.033376 22713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eb92e9162b347c8a590bdff6261e32d. Latest consensus state: current_term: 1 leader_uuid: "4eb92e9162b347c8a590bdff6261e32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } } }
I20260807 15:06:03.033499 22714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eb92e9162b347c8a590bdff6261e32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb92e9162b347c8a590bdff6261e32d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38067 } } }
I20260807 15:06:03.034495 22713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [sys.catalog]: This master's current role is: LEADER
W20260807 15:06:03.033821 22726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:06:03.034499 22714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d [sys.catalog]: This master's current role is: LEADER
W20260807 15:06:03.034600 22726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:03.034924 22729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:03.035257 22729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:03.036516 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:03.022514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55017 (local address 127.0.0.1:38067)
0807 15:06:03.022779 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:03.022791 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:03.022827 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:06:03.023399 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:03.023409 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:03.023479 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:03.023668 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:03.024413 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:03.026170 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:03.027184 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:03.027926 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:03.028073 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:03.031574 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:03.031579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:03.031579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:03.031594 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:03.034057 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:03.034848 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:03.034852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:03.034853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:03.034985 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:03.035301 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:03.035302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:03.035303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:03.035723 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:03.035913 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:03.036086 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:06:03.036258 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260807 15:06:03.037510 22729 catalog_manager.cc:1347] Generated new cluster ID: 936e5d954d6b45ef8fdaf57cc913d45d
I20260807 15:06:03.037567 22729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:03.037731 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:03.044441 22729 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:03.045252 22729 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:03.052572 22729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eb92e9162b347c8a590bdff6261e32d: Generated new TSK 0
I20260807 15:06:03.052804 22729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:03.053105 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:03.044248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51559 (local address 127.0.0.1:38067)
0807 15:06:03.044293 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:03.044296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:03.044400 (+ 104us) server_negotiation.cc:407] Connection header received
0807 15:06:03.044526 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:03.044527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:03.044539 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:03.044608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:03.045385 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:03.046162 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:03.047354 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:03.047717 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:03.047796 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:03.049493 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:03.049495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:03.049496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:03.049504 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:03.051326 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:03.051848 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:03.051849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:03.051850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:03.051933 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:03.052282 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:03.052284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:03.052284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:03.052438 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:03.052536 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:03.052928 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 15:06:03.053024 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:03.055446 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22637
W20260807 15:06:03.062697 22622 connection.cc:535] client connection to 127.0.0.1:38067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38067): Connection reset by peer (error 104)
2026-08-07T15:06:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:03 dist-test-slave-8tdl krb5kdc[22738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:03 dist-test-slave-8tdl krb5kdc[22738](info): set up 2 sockets
Aug 07 15:06:03 dist-test-slave-8tdl krb5kdc[22738](info): commencing operation
krb5kdc: starting...
W20260807 15:06:04.193113 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:04 dist-test-slave-8tdl krb5kdc[22738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:04.343443 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:04.439064 22753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:04.439320 22753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:04.439347 22753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:04.440732 22753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:04.440789 22753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:04.440804 22753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:04.440819 22753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:04.440831 22753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:04.442610 22753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:04.442965 22753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:04.443244 22753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:04.448909 22759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:04.448909 22760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:04.449050 22762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:04.449867 22753 server_base.cc:1034] running on GCE node
I20260807 15:06:04.450302 22753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:04.450789 22753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:04.452013 22753 hybrid_clock.cc:648] HybridClock initialized: now 1786115164451975 us; error 52 us; skew 500 ppm
Aug 07 15:06:04 dist-test-slave-8tdl krb5kdc[22738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:04.455183 22753 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:04.455752 22753 webserver.cc:460] Webserver started at http://127.0.0.1:41529/ using document root <none> and password file <none>
I20260807 15:06:04.456015 22753 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:04.456089 22753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:04.456248 22753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:04.457442 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "b35fb909e0b64aa1b4f4e13393362812"
format_stamp: "Formatted at 2026-08-07 15:06:04 on dist-test-slave-8tdl"
server_key: "b56190f0864ac419da887257007f91ad"
server_key_iv: "7963f609245d85611ec4b88da825a42a"
server_key_version: "encryptionkey@0"
I20260807 15:06:04.457882 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "b35fb909e0b64aa1b4f4e13393362812"
format_stamp: "Formatted at 2026-08-07 15:06:04 on dist-test-slave-8tdl"
server_key: "b56190f0864ac419da887257007f91ad"
server_key_iv: "7963f609245d85611ec4b88da825a42a"
server_key_version: "encryptionkey@0"
I20260807 15:06:04.459851 22753 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:06:04.460793 22769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:04.461114 22753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:04.461212 22753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "b35fb909e0b64aa1b4f4e13393362812"
format_stamp: "Formatted at 2026-08-07 15:06:04 on dist-test-slave-8tdl"
server_key: "b56190f0864ac419da887257007f91ad"
server_key_iv: "7963f609245d85611ec4b88da825a42a"
server_key_version: "encryptionkey@0"
I20260807 15:06:04.461273 22753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:04.483696 22753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:04.486563 22753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:04.486773 22753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:04.491662 22753 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37323
I20260807 15:06:04.491703 22821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37323 every 8 connection(s)
I20260807 15:06:04.492120 22753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:04.493083 22822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:04.495755 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812: Bootstrap starting.
I20260807 15:06:04.496421 22822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:04.496848 22822 log.cc:824] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:04.497823 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812: No bootstrap required, opened a new log
I20260807 15:06:04.499751 22822 raft_consensus.cc:348] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260807 15:06:04.499922 22822 raft_consensus.cc:374] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:04.499948 22822 raft_consensus.cc:729] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b35fb909e0b64aa1b4f4e13393362812, State: Initialized, Role: FOLLOWER
I20260807 15:06:04.500113 22822 consensus_queue.cc:260] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260807 15:06:04.500205 22822 raft_consensus.cc:388] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:04.500243 22822 raft_consensus.cc:482] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:04.500319 22822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:04.501331 22822 raft_consensus.cc:504] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260807 15:06:04.501467 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22753
I20260807 15:06:04.501468 22822 leader_election.cc:302] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b35fb909e0b64aa1b4f4e13393362812; no voters:
I20260807 15:06:04.501587 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:04.501670 22822 leader_election.cc:288] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:04.501732 22825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:04.501829 1535 external_mini_cluster.cc:1428] Setting key 9f4bbadaac60ee33f0a2587d2a55bb87
I20260807 15:06:04.501933 22825 raft_consensus.cc:686] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [term 1 LEADER]: Becoming Leader. State: Replica: b35fb909e0b64aa1b4f4e13393362812, State: Running, Role: LEADER
I20260807 15:06:04.501998 22822 sys_catalog.cc:564] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:04.502115 22825 consensus_queue.cc:237] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260807 15:06:04.502615 22828 sys_catalog.cc:455] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b35fb909e0b64aa1b4f4e13393362812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } } }
I20260807 15:06:04.502775 22828 sys_catalog.cc:458] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:04.502609 22827 sys_catalog.cc:455] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b35fb909e0b64aa1b4f4e13393362812. Latest consensus state: current_term: 1 leader_uuid: "b35fb909e0b64aa1b4f4e13393362812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b35fb909e0b64aa1b4f4e13393362812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } } }
I20260807 15:06:04.503218 22827 sys_catalog.cc:458] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:04.503626 22837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:04.504333 22837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:04.506183 22837 catalog_manager.cc:1347] Generated new cluster ID: e9e834146364418491b88f524d3a3f47
I20260807 15:06:04.506259 22837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:04.508550 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:04.502362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51051 (local address 127.0.0.1:37323)
0807 15:06:04.502658 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:04.502695 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:04.502883 (+ 188us) server_negotiation.cc:407] Connection header received
0807 15:06:04.503032 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:04.503040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:04.503111 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:04.503543 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:04.503676 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:04.505167 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:04.506175 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:04.506821 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:04.506964 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:04.507768 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:04.507772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:04.507772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:04.507981 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:04.508015 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:04.508028 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:04.508131 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:04.508219 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:06:04.508253 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260807 15:06:04.509711 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:04.514485 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:04.511257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52437 (local address 127.0.0.1:37323)
0807 15:06:04.511314 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:04.511316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:04.511452 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:06:04.511531 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:04.511532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:04.511544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:04.511600 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:04.511755 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:04.512304 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:04.513077 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:04.513333 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:04.513416 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:04.514196 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:04.514197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:04.514197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:04.514203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:04.514219 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:04.514224 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:04.514308 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:04.514376 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 15:06:04.514392 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:06:04.516364 22837 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:04.517138 22837 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:04.524524 22837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b35fb909e0b64aa1b4f4e13393362812: Generated new TSK 0
I20260807 15:06:04.524801 22837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:04.532598 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22753
W20260807 15:06:04.539584 22736 connection.cc:535] client connection to 127.0.0.1:37323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37323): Connection reset by peer (error 104)
2026-08-07T15:06:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:04 dist-test-slave-8tdl krb5kdc[22856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:04 dist-test-slave-8tdl krb5kdc[22856](info): set up 2 sockets
Aug 07 15:06:04 dist-test-slave-8tdl krb5kdc[22856](info): commencing operation
krb5kdc: starting...
W20260807 15:06:05.703589 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:05 dist-test-slave-8tdl krb5kdc[22856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:05.852586 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:05.944130 22869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:05.944471 22869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:05.944517 22869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:05.946235 22869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:05.946525 22869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:05.946601 22869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:05.946633 22869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:05.946667 22869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:05.949018 22869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:05.949427 22869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:05.949735 22869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:05.956599 22874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:05.956588 22875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:05.956588 22878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:05.957120 22869 server_base.cc:1034] running on GCE node
I20260807 15:06:05.957451 22869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:05.957839 22869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:05.959133 22869 hybrid_clock.cc:648] HybridClock initialized: now 1786115165959093 us; error 57 us; skew 500 ppm
Aug 07 15:06:05 dist-test-slave-8tdl krb5kdc[22856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115165, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:05.962193 22869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:05.962987 22869 webserver.cc:460] Webserver started at http://127.0.0.1:34995/ using document root <none> and password file <none>
I20260807 15:06:05.963258 22869 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:05.963337 22869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:05.963498 22869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:05.964689 22869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "a48306c88af34fb5bf77a8b338852a91"
format_stamp: "Formatted at 2026-08-07 15:06:05 on dist-test-slave-8tdl"
server_key: "f2d372b9440c92032bb6b0cb7ace6465"
server_key_iv: "c302850944f9951dc5bc5cef9525d6d3"
server_key_version: "encryptionkey@0"
I20260807 15:06:05.965241 22869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "a48306c88af34fb5bf77a8b338852a91"
format_stamp: "Formatted at 2026-08-07 15:06:05 on dist-test-slave-8tdl"
server_key: "f2d372b9440c92032bb6b0cb7ace6465"
server_key_iv: "c302850944f9951dc5bc5cef9525d6d3"
server_key_version: "encryptionkey@0"
I20260807 15:06:05.967453 22869 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260807 15:06:05.968540 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:05.968807 22869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:05.968977 22869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "a48306c88af34fb5bf77a8b338852a91"
format_stamp: "Formatted at 2026-08-07 15:06:05 on dist-test-slave-8tdl"
server_key: "f2d372b9440c92032bb6b0cb7ace6465"
server_key_iv: "c302850944f9951dc5bc5cef9525d6d3"
server_key_version: "encryptionkey@0"
I20260807 15:06:05.969108 22869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:05.972853 22869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:05.975826 22869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:05.976032 22869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:05.980688 22869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34737
I20260807 15:06:05.980741 22940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34737 every 8 connection(s)
I20260807 15:06:05.981159 22869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:05.982272 22942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:05.984803 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: Bootstrap starting.
I20260807 15:06:05.985800 22942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:05.986277 22942 log.cc:824] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:05.987368 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: No bootstrap required, opened a new log
I20260807 15:06:05.989233 22942 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } }
I20260807 15:06:05.989427 22942 raft_consensus.cc:374] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:05.989481 22942 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48306c88af34fb5bf77a8b338852a91, State: Initialized, Role: FOLLOWER
I20260807 15:06:05.989663 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22869
I20260807 15:06:05.989671 22942 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } }
I20260807 15:06:05.989758 22942 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:05.989800 22942 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:05.989806 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:05.989856 22942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:05.990006 1535 external_mini_cluster.cc:1428] Setting key d8f958936e26b829019c9ae150e44e4f
I20260807 15:06:05.990593 22942 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } }
I20260807 15:06:05.990782 22942 leader_election.cc:302] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48306c88af34fb5bf77a8b338852a91; no voters:
I20260807 15:06:05.990953 22942 leader_election.cc:288] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:05.991030 22947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:05.991212 22947 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [term 1 LEADER]: Becoming Leader. State: Replica: a48306c88af34fb5bf77a8b338852a91, State: Running, Role: LEADER
I20260807 15:06:05.991256 22942 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:05.991386 22947 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } }
I20260807 15:06:05.992189 22948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48306c88af34fb5bf77a8b338852a91. Latest consensus state: current_term: 1 leader_uuid: "a48306c88af34fb5bf77a8b338852a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } } }
I20260807 15:06:05.992296 22948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:05.992532 22949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a48306c88af34fb5bf77a8b338852a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48306c88af34fb5bf77a8b338852a91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34737 } } }
I20260807 15:06:05.992657 22949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:05.992731 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:05.990605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41449 (local address 127.0.0.1:34737)
0807 15:06:05.990833 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:05.990844 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:05.990895 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:06:05.991033 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:05.991041 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:05.991077 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:05.991255 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:05.991984 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:05.991988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:05.991989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:05.991994 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:05.992050 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:05.992068 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:05.992143 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:05.992401 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 15:06:05.992452 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
W20260807 15:06:05.993840 22963 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:06:05.993870 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 15:06:05.993896 22963 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:05.993948 22956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:05.994136 22956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:05.995836 22956 catalog_manager.cc:1347] Generated new cluster ID: 8969cce92868479180909bfaf758ee01
I20260807 15:06:05.995905 22956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:06.000536 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:05.999099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51233 (local address 127.0.0.1:34737)
0807 15:06:05.999165 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:05.999168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:05.999446 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:06:05.999518 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:05.999519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:05.999526 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:05.999583 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:06.000269 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:06.000271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:06.000271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:06.000277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:06.000295 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:06.000300 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:06.000355 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:06.000421 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:06:06.000439 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:06:06.008193 22956 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:06.008421 22956 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:06.023918 22956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a48306c88af34fb5bf77a8b338852a91: Generated new TSK 0
I20260807 15:06:06.024204 22956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:06.027221 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22869
2026-08-07T15:06:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:06 dist-test-slave-8tdl krb5kdc[22974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:06 dist-test-slave-8tdl krb5kdc[22974](info): set up 2 sockets
Aug 07 15:06:06 dist-test-slave-8tdl krb5kdc[22974](info): commencing operation
krb5kdc: starting...
W20260807 15:06:07.151176 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[22974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:07.299139 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35325
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:07.389786 22988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:07.390070 22988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:07.390116 22988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:07.391530 22988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:07.391613 22988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:07.391640 22988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:07.391673 22988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:07.391697 22988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:07.393353 22988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:07.393702 22988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:07.393980 22988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:07.400022 22997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:07.400141 22988 server_base.cc:1034] running on GCE node
W20260807 15:06:07.400027 22994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:07.400096 22995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:07.400681 22988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:07.401067 22988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:07.402282 22988 hybrid_clock.cc:648] HybridClock initialized: now 1786115167402258 us; error 42 us; skew 500 ppm
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[22974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:07.405553 22988 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:07.406111 22988 webserver.cc:460] Webserver started at http://127.0.0.1:41935/ using document root <none> and password file <none>
I20260807 15:06:07.406368 22988 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:07.406450 22988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:07.406603 22988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:07.407766 22988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "d6566b61f2b24f0cb6acd8a3c375761c"
format_stamp: "Formatted at 2026-08-07 15:06:07 on dist-test-slave-8tdl"
server_key: "6f11afe6ef8862cd791c1e9612c8c0f9"
server_key_iv: "227482a9c6cd619f528760840b547d8b"
server_key_version: "encryptionkey@0"
I20260807 15:06:07.408157 22988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "d6566b61f2b24f0cb6acd8a3c375761c"
format_stamp: "Formatted at 2026-08-07 15:06:07 on dist-test-slave-8tdl"
server_key: "6f11afe6ef8862cd791c1e9612c8c0f9"
server_key_iv: "227482a9c6cd619f528760840b547d8b"
server_key_version: "encryptionkey@0"
I20260807 15:06:07.409667 22988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:07.410602 23004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:07.410848 22988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:07.410976 22988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "d6566b61f2b24f0cb6acd8a3c375761c"
format_stamp: "Formatted at 2026-08-07 15:06:07 on dist-test-slave-8tdl"
server_key: "6f11afe6ef8862cd791c1e9612c8c0f9"
server_key_iv: "227482a9c6cd619f528760840b547d8b"
server_key_version: "encryptionkey@0"
I20260807 15:06:07.411311 22988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:07.431422 22988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:07.434131 22988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:07.434357 22988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:07.439459 22988 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35325
I20260807 15:06:07.439504 23058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35325 every 8 connection(s)
I20260807 15:06:07.439903 22988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:07.440945 23059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:07.443563 23059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c: Bootstrap starting.
I20260807 15:06:07.444213 23059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:07.444535 23059 log.cc:824] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:07.445474 23059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c: No bootstrap required, opened a new log
I20260807 15:06:07.446173 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 22988
I20260807 15:06:07.446345 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:07.446561 1535 external_mini_cluster.cc:1428] Setting key 453b85ccc5a248e7533634bc38e2ead3
I20260807 15:06:07.447779 23059 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260807 15:06:07.447970 23059 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:07.448017 23059 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6566b61f2b24f0cb6acd8a3c375761c, State: Initialized, Role: FOLLOWER
I20260807 15:06:07.448151 23059 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260807 15:06:07.448269 23059 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:07.448369 23059 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:07.448455 23059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:07.449599 23059 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260807 15:06:07.449747 23059 leader_election.cc:302] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6566b61f2b24f0cb6acd8a3c375761c; no voters:
I20260807 15:06:07.449982 23059 leader_election.cc:288] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:07.450032 23065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:07.450197 23065 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [term 1 LEADER]: Becoming Leader. State: Replica: d6566b61f2b24f0cb6acd8a3c375761c, State: Running, Role: LEADER
I20260807 15:06:07.450292 23059 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:07.450380 23065 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } }
I20260807 15:06:07.450774 23065 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6566b61f2b24f0cb6acd8a3c375761c. Latest consensus state: current_term: 1 leader_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } } }
I20260807 15:06:07.450930 23065 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:07.451424 23074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:07.451685 23067 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6566b61f2b24f0cb6acd8a3c375761c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35325 } } }
I20260807 15:06:07.451773 23067 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:07.452131 23074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:07.454181 23074 catalog_manager.cc:1347] Generated new cluster ID: 890be54ff4a24d26ac8722f8b806bb50
I20260807 15:06:07.454242 23074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[22974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:06:07.459757 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:07.447215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49663 (local address 127.0.0.1:35325)
0807 15:06:07.447432 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:07.447441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:07.447471 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:06:07.447792 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:07.447800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:07.447872 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:07.448052 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:07.449163 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:07.450347 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:07.451463 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:07.452135 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:07.452257 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:07.455506 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:07.455511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:07.455511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:07.455533 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:07.457739 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:07.458254 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:07.458258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:07.458258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:07.458394 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:07.458647 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:07.458648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:07.458648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:07.459037 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:07.459234 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:07.459307 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:06:07.459585 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":125,"threads_started":1}
I20260807 15:06:07.460755 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:07.469285 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:07.462286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60273 (local address 127.0.0.1:35325)
0807 15:06:07.462325 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:07.462327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:07.462858 (+ 531us) server_negotiation.cc:407] Connection header received
0807 15:06:07.462934 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:07.462935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:07.462946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:07.462990 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:07.463592 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:07.464084 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:07.464825 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:07.465055 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:07.465137 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:07.466448 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:07.466449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:07.466449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:07.466458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:07.468017 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:07.468407 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:07.468408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:07.468408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:07.468476 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:07.468691 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:07.468692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:07.468692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:07.468817 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:07.468933 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:07.469068 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:06:07.469160 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:07.483341 23074 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:07.484148 23074 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:07.488867 23074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6566b61f2b24f0cb6acd8a3c375761c: Generated new TSK 0
I20260807 15:06:07.489094 23074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:07.495198 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 22988
2026-08-07T15:06:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[23091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[23091](info): set up 2 sockets
Aug 07 15:06:07 dist-test-slave-8tdl krb5kdc[23091](info): commencing operation
krb5kdc: starting...
W20260807 15:06:08.603821 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:08 dist-test-slave-8tdl krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:08.756651 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:08.851320 23104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:08.851629 23104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:08.851677 23104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:08.853081 23104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:08.853153 23104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:08.853192 23104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:08.853221 23104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:08.853245 23104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:08.855059 23104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:08.855399 23104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:08.855661 23104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:08.862463 23109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:08.862468 23110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:08.862488 23104 server_base.cc:1034] running on GCE node
W20260807 15:06:08.862464 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:08.863101 23104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:08.863505 23104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:08.864717 23104 hybrid_clock.cc:648] HybridClock initialized: now 1786115168864686 us; error 56 us; skew 500 ppm
Aug 07 15:06:08 dist-test-slave-8tdl krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115168, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:08.877525 23104 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:08.878185 23104 webserver.cc:460] Webserver started at http://127.0.0.1:42507/ using document root <none> and password file <none>
I20260807 15:06:08.878490 23104 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:08.878569 23104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:08.878782 23104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:08.880154 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "2e7ff7416c6448a9ba273b51881d5d40"
format_stamp: "Formatted at 2026-08-07 15:06:08 on dist-test-slave-8tdl"
server_key: "4c3984b5f37934aa8982f3d56ef5a87b"
server_key_iv: "5ea0084331e3b7c80a665c441f5b90e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:08.880616 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "2e7ff7416c6448a9ba273b51881d5d40"
format_stamp: "Formatted at 2026-08-07 15:06:08 on dist-test-slave-8tdl"
server_key: "4c3984b5f37934aa8982f3d56ef5a87b"
server_key_iv: "5ea0084331e3b7c80a665c441f5b90e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:08.882503 23104 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:06:08.884016 23121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:08.884495 23104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:08.884671 23104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "2e7ff7416c6448a9ba273b51881d5d40"
format_stamp: "Formatted at 2026-08-07 15:06:08 on dist-test-slave-8tdl"
server_key: "4c3984b5f37934aa8982f3d56ef5a87b"
server_key_iv: "5ea0084331e3b7c80a665c441f5b90e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:08.884786 23104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:08.901562 23104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:08.904249 23104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:08.904450 23104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:08.909524 23104 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38967
I20260807 15:06:08.909560 23177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38967 every 8 connection(s)
I20260807 15:06:08.910024 23104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:08.911576 23178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:08.914453 23178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: Bootstrap starting.
I20260807 15:06:08.914893 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23104
I20260807 15:06:08.915050 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:08.915256 1535 external_mini_cluster.cc:1428] Setting key 6613ae9fd9531e80a3a8d9ff44df8251
I20260807 15:06:08.915565 23178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:08.915979 23178 log.cc:824] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:08.916816 23178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: No bootstrap required, opened a new log
I20260807 15:06:08.919380 23178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260807 15:06:08.919607 23178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:08.919642 23178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e7ff7416c6448a9ba273b51881d5d40, State: Initialized, Role: FOLLOWER
I20260807 15:06:08.919785 23178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260807 15:06:08.919888 23178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:08.919952 23178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:08.920008 23178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:08.920842 23178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260807 15:06:08.921089 23178 leader_election.cc:302] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e7ff7416c6448a9ba273b51881d5d40; no voters:
I20260807 15:06:08.921612 23178 leader_election.cc:288] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:08.921996 23185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:08.922072 23178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:08.922298 23185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [term 1 LEADER]: Becoming Leader. State: Replica: 2e7ff7416c6448a9ba273b51881d5d40, State: Running, Role: LEADER
I20260807 15:06:08.922572 23185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } }
I20260807 15:06:08.922873 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:08.916075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52485 (local address 127.0.0.1:38967)
0807 15:06:08.916301 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:08.916309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:08.916337 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:06:08.917123 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:08.917132 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:08.917193 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:08.917326 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:08.917663 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:08.918635 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:08.919862 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:08.920442 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:08.920551 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:08.921679 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:08.921684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:08.921684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:08.922123 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:08.922180 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:08.922200 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:08.922273 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:08.922544 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 15:06:08.922629 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260807 15:06:08.923936 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:08.924180 23186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e7ff7416c6448a9ba273b51881d5d40. Latest consensus state: current_term: 1 leader_uuid: "2e7ff7416c6448a9ba273b51881d5d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } } }
I20260807 15:06:08.924245 23184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e7ff7416c6448a9ba273b51881d5d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7ff7416c6448a9ba273b51881d5d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38967 } } }
I20260807 15:06:08.924293 23184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:08.924293 23186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40 [sys.catalog]: This master's current role is: LEADER
W20260807 15:06:08.924685 23199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:08.925570 23199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:08.925663 23200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:08.925935 23200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:08.928184 23200 catalog_manager.cc:1347] Generated new cluster ID: 22fce63e882746da910a0cb41654df42
I20260807 15:06:08.928275 23200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:08.929795 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:08.925608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51449 (local address 127.0.0.1:38967)
0807 15:06:08.925675 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:08.925677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:08.925893 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:06:08.925930 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:08.925931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:08.925961 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:08.926037 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:08.926301 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:08.927109 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:08.928212 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:08.928516 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:08.928564 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:08.929493 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:08.929495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:08.929495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:08.929502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:08.929519 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:08.929524 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:08.929553 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:08.929667 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:06:08.929704 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:06:08.952410 23200 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:08.953331 23200 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:08.959734 23200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e7ff7416c6448a9ba273b51881d5d40: Generated new TSK 0
I20260807 15:06:08.960052 23200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:08.968834 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23104
2026-08-07T15:06:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:09 dist-test-slave-8tdl krb5kdc[23210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:09 dist-test-slave-8tdl krb5kdc[23210](info): set up 2 sockets
Aug 07 15:06:09 dist-test-slave-8tdl krb5kdc[23210](info): commencing operation
krb5kdc: starting...
W20260807 15:06:10.114089 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:10 dist-test-slave-8tdl krb5kdc[23210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:10.276386 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:10.374936 23224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:10.375291 23224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:10.375348 23224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:10.376766 23224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:10.376827 23224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:10.376844 23224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:10.376858 23224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:10.376873 23224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:10.378825 23224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:10.379169 23224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:10.379447 23224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:10.385455 23229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:10.385455 23230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:10.385672 23232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:10.385941 23224 server_base.cc:1034] running on GCE node
I20260807 15:06:10.386284 23224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:10.386763 23224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:10.388005 23224 hybrid_clock.cc:648] HybridClock initialized: now 1786115170387966 us; error 67 us; skew 500 ppm
Aug 07 15:06:10 dist-test-slave-8tdl krb5kdc[23210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:10.392022 23224 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:10.392695 23224 webserver.cc:460] Webserver started at http://127.0.0.1:32951/ using document root <none> and password file <none>
I20260807 15:06:10.393047 23224 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:10.393165 23224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:10.393354 23224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:10.395382 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "e16eced3b95e44c095b68e61861083dc"
format_stamp: "Formatted at 2026-08-07 15:06:10 on dist-test-slave-8tdl"
server_key: "20b949d678e43660ce08d027160edfbf"
server_key_iv: "9852055e2048d7e5404071bc102a66b9"
server_key_version: "encryptionkey@0"
I20260807 15:06:10.395928 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "e16eced3b95e44c095b68e61861083dc"
format_stamp: "Formatted at 2026-08-07 15:06:10 on dist-test-slave-8tdl"
server_key: "20b949d678e43660ce08d027160edfbf"
server_key_iv: "9852055e2048d7e5404071bc102a66b9"
server_key_version: "encryptionkey@0"
I20260807 15:06:10.397842 23224 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:06:10.398813 23239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:10.399060 23224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:10.399183 23224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e16eced3b95e44c095b68e61861083dc"
format_stamp: "Formatted at 2026-08-07 15:06:10 on dist-test-slave-8tdl"
server_key: "20b949d678e43660ce08d027160edfbf"
server_key_iv: "9852055e2048d7e5404071bc102a66b9"
server_key_version: "encryptionkey@0"
I20260807 15:06:10.399290 23224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:10.403492 23224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:10.406162 23224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:10.406359 23224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:10.412241 23224 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33695
I20260807 15:06:10.412302 23296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33695 every 8 connection(s)
I20260807 15:06:10.412750 23224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:10.413920 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23224
I20260807 15:06:10.414048 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:10.414075 23297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:10.414274 1535 external_mini_cluster.cc:1428] Setting key 0a9363fc52ce1c4ae422fa0d3c24f595
I20260807 15:06:10.417497 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:10.415052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46751 (local address 127.0.0.1:33695)
0807 15:06:10.415284 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:10.415292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:10.415359 (+ 67us) server_negotiation.cc:407] Connection header received
0807 15:06:10.415884 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:10.415892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:10.415923 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:10.416068 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:10.416912 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:10.416917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:10.416917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:10.416923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:10.416983 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:10.416996 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:10.417072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:10.417168 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 15:06:10.417246 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260807 15:06:10.418228 23297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc: Bootstrap starting.
I20260807 15:06:10.418789 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:10.419108 23297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:10.419420 23297 log.cc:824] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:10.420326 23297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc: No bootstrap required, opened a new log
I20260807 15:06:10.421816 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:10.420382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55941 (local address 127.0.0.1:33695)
0807 15:06:10.420429 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:10.420431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:10.420536 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:06:10.420608 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:10.420608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:10.420614 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:10.420662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:10.421581 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:10.421582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:10.421583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:10.421587 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:10.421603 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:10.421609 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:10.421662 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:10.421727 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 15:06:10.421744 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:10.422513 23297 raft_consensus.cc:348] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260807 15:06:10.422715 23297 raft_consensus.cc:374] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:10.422792 23297 raft_consensus.cc:729] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16eced3b95e44c095b68e61861083dc, State: Initialized, Role: FOLLOWER
I20260807 15:06:10.422951 23297 consensus_queue.cc:260] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260807 15:06:10.423048 23297 raft_consensus.cc:388] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:10.423094 23297 raft_consensus.cc:482] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:10.423148 23297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:10.423816 23297 raft_consensus.cc:504] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260807 15:06:10.423957 23297 leader_election.cc:302] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e16eced3b95e44c095b68e61861083dc; no voters:
I20260807 15:06:10.424175 23297 leader_election.cc:288] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:10.424328 23309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:10.424467 23297 sys_catalog.cc:564] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:10.424535 23309 raft_consensus.cc:686] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [term 1 LEADER]: Becoming Leader. State: Replica: e16eced3b95e44c095b68e61861083dc, State: Running, Role: LEADER
I20260807 15:06:10.424660 23309 consensus_queue.cc:237] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260807 15:06:10.425159 23310 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e16eced3b95e44c095b68e61861083dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260807 15:06:10.425169 23311 sys_catalog.cc:455] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e16eced3b95e44c095b68e61861083dc. Latest consensus state: current_term: 1 leader_uuid: "e16eced3b95e44c095b68e61861083dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e16eced3b95e44c095b68e61861083dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260807 15:06:10.425310 23310 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:10.425319 23311 sys_catalog.cc:458] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:10.425580 23318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:10.426126 23318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:10.428224 23318 catalog_manager.cc:1347] Generated new cluster ID: 1988f89ac6504f92b79ca28688756892
I20260807 15:06:10.428297 23318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:10.441804 23318 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:10.442190 23318 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:10.449571 23318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e16eced3b95e44c095b68e61861083dc: Generated new TSK 0
I20260807 15:06:10.449903 23318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:10.460034 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23224
2026-08-07T15:06:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:10 dist-test-slave-8tdl krb5kdc[23331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:10 dist-test-slave-8tdl krb5kdc[23331](info): set up 2 sockets
Aug 07 15:06:10 dist-test-slave-8tdl krb5kdc[23331](info): commencing operation
krb5kdc: starting...
W20260807 15:06:11.602659 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:11 dist-test-slave-8tdl krb5kdc[23331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:11.768561 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:11.863091 23344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:11.863340 23344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:11.863371 23344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:11.864768 23344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:11.864830 23344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:11.864851 23344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:11.864866 23344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:11.864878 23344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:11.866827 23344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:11.867147 23344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:11.867420 23344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:11.873008 23350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:11.873203 23352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:11.873410 23344 server_base.cc:1034] running on GCE node
W20260807 15:06:11.873176 23349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:11.873756 23344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:11.874100 23344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:11.875283 23344 hybrid_clock.cc:648] HybridClock initialized: now 1786115171875235 us; error 66 us; skew 500 ppm
Aug 07 15:06:11 dist-test-slave-8tdl krb5kdc[23331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115171, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:11.878403 23344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:11.879056 23344 webserver.cc:460] Webserver started at http://127.0.0.1:35385/ using document root <none> and password file <none>
I20260807 15:06:11.879333 23344 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:11.879410 23344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:11.879573 23344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:11.880882 23344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab"
format_stamp: "Formatted at 2026-08-07 15:06:11 on dist-test-slave-8tdl"
server_key: "3d1b3a3e4f51de18f76476a37176c852"
server_key_iv: "38703f3bf73b04a3d1036a65d0d79fe1"
server_key_version: "encryptionkey@0"
I20260807 15:06:11.881305 23344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab"
format_stamp: "Formatted at 2026-08-07 15:06:11 on dist-test-slave-8tdl"
server_key: "3d1b3a3e4f51de18f76476a37176c852"
server_key_iv: "38703f3bf73b04a3d1036a65d0d79fe1"
server_key_version: "encryptionkey@0"
I20260807 15:06:11.883013 23344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:11.884052 23359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:11.884294 23344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:11.884421 23344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab"
format_stamp: "Formatted at 2026-08-07 15:06:11 on dist-test-slave-8tdl"
server_key: "3d1b3a3e4f51de18f76476a37176c852"
server_key_iv: "38703f3bf73b04a3d1036a65d0d79fe1"
server_key_version: "encryptionkey@0"
I20260807 15:06:11.884513 23344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:11.906225 23344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:11.909204 23344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:11.909440 23344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:11.914621 23344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34659
I20260807 15:06:11.914737 23414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34659 every 8 connection(s)
I20260807 15:06:11.915313 23344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:11.915710 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23344
I20260807 15:06:11.915923 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:11.916196 1535 external_mini_cluster.cc:1428] Setting key 17311014657bf432dd4e5c895b5ce278
I20260807 15:06:11.916630 23415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:11.920768 23415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab: Bootstrap starting.
I20260807 15:06:11.921654 23415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:11.922055 23415 log.cc:824] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:11.922951 23415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab: No bootstrap required, opened a new log
Aug 07 15:06:11 dist-test-slave-8tdl krb5kdc[23331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:06:11.925549 23415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260807 15:06:11.925726 23415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:11.925750 23415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f2fae44ac8c47679e10b02e1ba1f2ab, State: Initialized, Role: FOLLOWER
I20260807 15:06:11.925979 23415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260807 15:06:11.926090 23415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:11.926141 23415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:11.926213 23415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:11.927122 23415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260807 15:06:11.927345 23415 leader_election.cc:302] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f2fae44ac8c47679e10b02e1ba1f2ab; no voters:
I20260807 15:06:11.927930 23420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:11.927882 23415 leader_election.cc:288] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:11.928104 23420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [term 1 LEADER]: Becoming Leader. State: Replica: 4f2fae44ac8c47679e10b02e1ba1f2ab, State: Running, Role: LEADER
I20260807 15:06:11.928279 23420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } }
I20260807 15:06:11.928313 23415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:11.929143 23421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } } }
I20260807 15:06:11.929237 23421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:11.929834 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:11.917156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:43173 (local address 127.0.0.1:34659)
0807 15:06:11.917391 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:11.917405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:11.917435 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:06:11.917880 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:11.917888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:11.917944 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:11.918115 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:11.919054 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:11.920026 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:11.921153 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:11.921830 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:11.921967 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:11.925342 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:11.925347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:11.925347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:11.925361 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:11.927548 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:11.928137 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:11.928141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:11.928141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:11.928260 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:11.928574 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:11.928576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:11.928576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:11.928889 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:11.929060 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:11.929428 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 15:06:11.929569 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260807 15:06:11.929694 23422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f2fae44ac8c47679e10b02e1ba1f2ab. Latest consensus state: current_term: 1 leader_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2fae44ac8c47679e10b02e1ba1f2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34659 } } }
I20260807 15:06:11.929999 23422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:11.930102 23429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:11.931313 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:11.931340 23429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:11.933470 23429 catalog_manager.cc:1347] Generated new cluster ID: 7489eaea17074905b6be0a37520b51c8
I20260807 15:06:11.933539 23429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:11.941107 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:11.933036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:38027 (local address 127.0.0.1:34659)
0807 15:06:11.933073 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:11.933075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:11.933226 (+ 151us) server_negotiation.cc:407] Connection header received
0807 15:06:11.933289 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:11.933290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:11.933302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:11.933375 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:11.934079 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:11.934641 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:11.935554 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:11.935823 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:11.935924 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:11.937769 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:11.937771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:11.937771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:11.937781 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:11.939667 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:11.940187 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:11.940189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:11.940189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:11.940260 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:11.940512 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:11.940514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:11.940514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:11.940646 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:11.940802 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:11.940901 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:06:11.940982 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:06:11.964258 23429 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:11.965098 23429 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:11.974017 23429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f2fae44ac8c47679e10b02e1ba1f2ab: Generated new TSK 0
I20260807 15:06:11.974308 23429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:11.981397 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23344
W20260807 15:06:11.988492 23329 connection.cc:535] client connection to 127.0.0.1:34659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34659): Connection reset by peer (error 104)
2026-08-07T15:06:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:12 dist-test-slave-8tdl krb5kdc[23448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:12 dist-test-slave-8tdl krb5kdc[23448](info): set up 2 sockets
Aug 07 15:06:12 dist-test-slave-8tdl krb5kdc[23448](info): commencing operation
krb5kdc: starting...
W20260807 15:06:13.091519 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:13 dist-test-slave-8tdl krb5kdc[23448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:13.234388 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:13.320626 23461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:13.320899 23461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:13.320950 23461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:13.322229 23461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:13.322310 23461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:13.322335 23461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:13.322366 23461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:13.322398 23461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:13.324033 23461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:13.324348 23461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:13.324600 23461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:13.329859 23467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:13.330139 23461 server_base.cc:1034] running on GCE node
W20260807 15:06:13.329838 23466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:13.329838 23469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:13.330514 23461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:13.330880 23461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:13.332111 23461 hybrid_clock.cc:648] HybridClock initialized: now 1786115173332102 us; error 36 us; skew 500 ppm
Aug 07 15:06:13 dist-test-slave-8tdl krb5kdc[23448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115173, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:13.335119 23461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:13.335649 23461 webserver.cc:460] Webserver started at http://127.0.0.1:37343/ using document root <none> and password file <none>
I20260807 15:06:13.335901 23461 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:13.335973 23461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:13.336136 23461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:13.337347 23461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "b56c38e3bbe4425393ce36ed082fdf30"
format_stamp: "Formatted at 2026-08-07 15:06:13 on dist-test-slave-8tdl"
server_key: "139beb5ba8b6e418c9025cb1097ed781"
server_key_iv: "e201ebe0b3ead3c8d5858741530da638"
server_key_version: "encryptionkey@0"
I20260807 15:06:13.337790 23461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "b56c38e3bbe4425393ce36ed082fdf30"
format_stamp: "Formatted at 2026-08-07 15:06:13 on dist-test-slave-8tdl"
server_key: "139beb5ba8b6e418c9025cb1097ed781"
server_key_iv: "e201ebe0b3ead3c8d5858741530da638"
server_key_version: "encryptionkey@0"
I20260807 15:06:13.339540 23461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:06:13.340678 23476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:13.340970 23461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:13.341092 23461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "b56c38e3bbe4425393ce36ed082fdf30"
format_stamp: "Formatted at 2026-08-07 15:06:13 on dist-test-slave-8tdl"
server_key: "139beb5ba8b6e418c9025cb1097ed781"
server_key_iv: "e201ebe0b3ead3c8d5858741530da638"
server_key_version: "encryptionkey@0"
I20260807 15:06:13.341189 23461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:13.356644 23461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:13.359181 23461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:13.359309 23461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:13.364070 23461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45305
I20260807 15:06:13.364218 23530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45305 every 8 connection(s)
I20260807 15:06:13.364549 23461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:13.365607 23531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:13.368101 23531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30: Bootstrap starting.
I20260807 15:06:13.368678 23531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:13.368937 23531 log.cc:824] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:13.369691 23531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30: No bootstrap required, opened a new log
I20260807 15:06:13.370921 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23461
I20260807 15:06:13.371027 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:13.371233 1535 external_mini_cluster.cc:1428] Setting key 39b1c171829cce32e328769b2354fdab
I20260807 15:06:13.371800 23531 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260807 15:06:13.372043 23531 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:13.372107 23531 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56c38e3bbe4425393ce36ed082fdf30, State: Initialized, Role: FOLLOWER
I20260807 15:06:13.372251 23531 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260807 15:06:13.372331 23531 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:13.372364 23531 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:13.372421 23531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:13.373750 23531 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260807 15:06:13.373862 23531 leader_election.cc:302] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56c38e3bbe4425393ce36ed082fdf30; no voters:
I20260807 15:06:13.374018 23531 leader_election.cc:288] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:13.374094 23536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:13.374245 23536 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [term 1 LEADER]: Becoming Leader. State: Replica: b56c38e3bbe4425393ce36ed082fdf30, State: Running, Role: LEADER
I20260807 15:06:13.374265 23531 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:13.374336 23536 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260807 15:06:13.374728 23536 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56c38e3bbe4425393ce36ed082fdf30. Latest consensus state: current_term: 1 leader_uuid: "b56c38e3bbe4425393ce36ed082fdf30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260807 15:06:13.374835 23536 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:13.374735 23537 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56c38e3bbe4425393ce36ed082fdf30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56c38e3bbe4425393ce36ed082fdf30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260807 15:06:13.375044 23537 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:13.375156 23544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:13.375825 23544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:13.377521 23544 catalog_manager.cc:1347] Generated new cluster ID: cd82abd24c314ae68a51ca6d0d8f0102
I20260807 15:06:13.377581 23544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:13.378147 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:13.371920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:52297 (local address 127.0.0.1:45305)
0807 15:06:13.372157 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:13.372167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:13.372198 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:06:13.372317 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:13.372324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:13.372387 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:13.372534 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:13.372755 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:13.374230 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:13.375540 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:13.376266 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:13.376740 (+ 474us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:13.377214 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:13.377218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:13.377218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:13.377449 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:13.377489 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:13.377504 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:13.377704 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:13.377794 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:06:13.377928 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260807 15:06:13.379139 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:13.384375 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:13.380669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:50537 (local address 127.0.0.1:45305)
0807 15:06:13.380705 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:13.380706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:13.380851 (+ 145us) server_negotiation.cc:407] Connection header received
0807 15:06:13.380881 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:13.380882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:13.380901 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:13.380967 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:13.381108 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:13.381841 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:13.382908 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:13.383308 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:13.383385 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:13.384104 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:13.384105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:13.384105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:13.384111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:13.384126 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:13.384131 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:13.384197 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:13.384271 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:06:13.384284 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:13.404282 23544 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:13.404920 23544 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:13.416381 23544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56c38e3bbe4425393ce36ed082fdf30: Generated new TSK 0
I20260807 15:06:13.416636 23544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:13.422341 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23461
W20260807 15:06:13.429975 23445 connection.cc:535] client connection to 127.0.0.1:45305 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45305): Connection reset by peer (error 104)
2026-08-07T15:06:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:13 dist-test-slave-8tdl krb5kdc[23564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:13 dist-test-slave-8tdl krb5kdc[23564](info): set up 2 sockets
Aug 07 15:06:13 dist-test-slave-8tdl krb5kdc[23564](info): commencing operation
krb5kdc: starting...
W20260807 15:06:14.509943 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:14 dist-test-slave-8tdl krb5kdc[23564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:14.650614 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:14.735772 23577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:14.736063 23577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:14.736107 23577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:14.737367 23577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:14.737452 23577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:14.737478 23577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:14.737502 23577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:14.737532 23577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:14.739181 23577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:14.739500 23577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:14.739758 23577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:14.745505 23583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:14.745599 23577 server_base.cc:1034] running on GCE node
W20260807 15:06:14.745505 23582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:14.745508 23585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:14.746197 23577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:14.746547 23577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:14.747740 23577 hybrid_clock.cc:648] HybridClock initialized: now 1786115174747716 us; error 42 us; skew 500 ppm
Aug 07 15:06:14 dist-test-slave-8tdl krb5kdc[23564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:14.756476 23577 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:14.757062 23577 webserver.cc:460] Webserver started at http://127.0.0.1:44385/ using document root <none> and password file <none>
I20260807 15:06:14.757369 23577 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:14.757484 23577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:14.757664 23577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:14.759277 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "6988ef2f2d014e5f9ec367f994441e15"
format_stamp: "Formatted at 2026-08-07 15:06:14 on dist-test-slave-8tdl"
server_key: "df04783c6f7ea0901560c2438e270209"
server_key_iv: "5c4b3d2a8fa11132fbdf2a68fcb3227e"
server_key_version: "encryptionkey@0"
I20260807 15:06:14.759727 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "6988ef2f2d014e5f9ec367f994441e15"
format_stamp: "Formatted at 2026-08-07 15:06:14 on dist-test-slave-8tdl"
server_key: "df04783c6f7ea0901560c2438e270209"
server_key_iv: "5c4b3d2a8fa11132fbdf2a68fcb3227e"
server_key_version: "encryptionkey@0"
I20260807 15:06:14.761432 23577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:14.762483 23595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:14.762813 23577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:14.763000 23577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "6988ef2f2d014e5f9ec367f994441e15"
format_stamp: "Formatted at 2026-08-07 15:06:14 on dist-test-slave-8tdl"
server_key: "df04783c6f7ea0901560c2438e270209"
server_key_iv: "5c4b3d2a8fa11132fbdf2a68fcb3227e"
server_key_version: "encryptionkey@0"
I20260807 15:06:14.763100 23577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:14.766861 23577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:14.769379 23577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:14.769557 23577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:14.774782 23577 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37703
I20260807 15:06:14.774827 23647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37703 every 8 connection(s)
I20260807 15:06:14.775244 23577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:14.776456 23648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:14.777077 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23577
I20260807 15:06:14.777206 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:14.777479 1535 external_mini_cluster.cc:1428] Setting key f52e521645548aba3f4ae869a40d2823
I20260807 15:06:14.780036 23648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15: Bootstrap starting.
I20260807 15:06:14.780536 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:14.778132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:45329 (local address 127.0.0.1:37703)
0807 15:06:14.778336 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:14.778344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:14.778393 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:06:14.778994 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:14.779002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:14.779031 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:14.779156 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:14.780006 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:14.780010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:14.780010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:14.780014 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:14.780052 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:14.780062 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:14.780176 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:14.780250 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:06:14.780310 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260807 15:06:14.780848 23648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:14.781225 23648 log.cc:824] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:14.781700 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:14.782159 23648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15: No bootstrap required, opened a new log
I20260807 15:06:14.784579 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:14.783299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:50611 (local address 127.0.0.1:37703)
0807 15:06:14.783346 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:14.783348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:14.783514 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:06:14.783542 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:14.783543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:14.783567 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:14.783644 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:14.784242 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:14.784244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:14.784244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:14.784250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:14.784272 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:14.784277 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:14.784361 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:14.784475 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:06:14.784491 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:06:14.785138 23648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } }
I20260807 15:06:14.785297 23648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:14.785320 23648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6988ef2f2d014e5f9ec367f994441e15, State: Initialized, Role: FOLLOWER
I20260807 15:06:14.785439 23648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } }
I20260807 15:06:14.785532 23648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:14.785555 23648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:14.785586 23648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:14.786213 23648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } }
I20260807 15:06:14.786307 23648 leader_election.cc:302] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6988ef2f2d014e5f9ec367f994441e15; no voters:
I20260807 15:06:14.786515 23648 leader_election.cc:288] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:14.786573 23658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:14.786815 23658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [term 1 LEADER]: Becoming Leader. State: Replica: 6988ef2f2d014e5f9ec367f994441e15, State: Running, Role: LEADER
I20260807 15:06:14.786823 23648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:14.786989 23658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } }
I20260807 15:06:14.787428 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6988ef2f2d014e5f9ec367f994441e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } } }
I20260807 15:06:14.787448 23660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6988ef2f2d014e5f9ec367f994441e15. Latest consensus state: current_term: 1 leader_uuid: "6988ef2f2d014e5f9ec367f994441e15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6988ef2f2d014e5f9ec367f994441e15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37703 } } }
I20260807 15:06:14.787621 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:14.787637 23660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:14.788070 23667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:14.788620 23667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:14.790292 23667 catalog_manager.cc:1347] Generated new cluster ID: 62cf3fe99fc8495d95dbf8d7fd11b90c
I20260807 15:06:14.790349 23667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:14.794449 23667 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:14.794705 23667 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:14.798341 23667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6988ef2f2d014e5f9ec367f994441e15: Generated new TSK 0
I20260807 15:06:14.798619 23667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:14.801729 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23577
2026-08-07T15:06:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:14 dist-test-slave-8tdl krb5kdc[23680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:14 dist-test-slave-8tdl krb5kdc[23680](info): set up 2 sockets
Aug 07 15:06:14 dist-test-slave-8tdl krb5kdc[23680](info): commencing operation
krb5kdc: starting...
W20260807 15:06:15.915664 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:16.054132 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:16.139160 23694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:16.139391 23694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:16.139417 23694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:16.140658 23694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:16.140715 23694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:16.140730 23694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:16.140743 23694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:16.140755 23694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:16.142395 23694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:16.142738 23694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:16.143033 23694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:16.148494 23700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:16.148739 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:16.148854 23694 server_base.cc:1034] running on GCE node
W20260807 15:06:16.149084 23699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:16.149441 23694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:16.149834 23694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:16.151062 23694 hybrid_clock.cc:648] HybridClock initialized: now 1786115176151024 us; error 49 us; skew 500 ppm
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:16.153987 23694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:16.154589 23694 webserver.cc:460] Webserver started at http://127.0.0.1:41543/ using document root <none> and password file <none>
I20260807 15:06:16.154888 23694 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:16.154969 23694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:16.155134 23694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:16.156301 23694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "6c0427b040404822ab3f4b22d3fe8b1f"
format_stamp: "Formatted at 2026-08-07 15:06:16 on dist-test-slave-8tdl"
server_key: "2aae3206d8e80b5eb62ed035691dabb6"
server_key_iv: "128f5d6baf311312d9255e7d108a359b"
server_key_version: "encryptionkey@0"
I20260807 15:06:16.156706 23694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "6c0427b040404822ab3f4b22d3fe8b1f"
format_stamp: "Formatted at 2026-08-07 15:06:16 on dist-test-slave-8tdl"
server_key: "2aae3206d8e80b5eb62ed035691dabb6"
server_key_iv: "128f5d6baf311312d9255e7d108a359b"
server_key_version: "encryptionkey@0"
I20260807 15:06:16.158252 23694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:16.159265 23709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:16.159565 23694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:06:16.159684 23694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "6c0427b040404822ab3f4b22d3fe8b1f"
format_stamp: "Formatted at 2026-08-07 15:06:16 on dist-test-slave-8tdl"
server_key: "2aae3206d8e80b5eb62ed035691dabb6"
server_key_iv: "128f5d6baf311312d9255e7d108a359b"
server_key_version: "encryptionkey@0"
I20260807 15:06:16.159771 23694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:16.179191 23694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:16.181890 23694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:16.182176 23694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:16.186592 23694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37243
I20260807 15:06:16.186650 23763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37243 every 8 connection(s)
I20260807 15:06:16.187083 23694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:16.188270 23764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:16.190421 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23694
I20260807 15:06:16.190578 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:16.190810 1535 external_mini_cluster.cc:1428] Setting key 0084182cf2c221749c04fa1f4337819c
I20260807 15:06:16.190979 23764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f: Bootstrap starting.
I20260807 15:06:16.191795 23764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:16.192118 23764 log.cc:824] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:16.193169 23764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f: No bootstrap required, opened a new log
I20260807 15:06:16.195581 23764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } }
I20260807 15:06:16.195868 23764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:16.195948 23764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0427b040404822ab3f4b22d3fe8b1f, State: Initialized, Role: FOLLOWER
I20260807 15:06:16.196139 23764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } }
I20260807 15:06:16.196226 23764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:16.196251 23764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:16.196326 23764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:16.197121 23764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } }
I20260807 15:06:16.197230 23764 leader_election.cc:302] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c0427b040404822ab3f4b22d3fe8b1f; no voters:
I20260807 15:06:16.197463 23764 leader_election.cc:288] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:16.197530 23769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:16.197706 23769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [term 1 LEADER]: Becoming Leader. State: Replica: 6c0427b040404822ab3f4b22d3fe8b1f, State: Running, Role: LEADER
I20260807 15:06:16.197769 23764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:16.197849 23769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } }
I20260807 15:06:16.198272 23770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } } }
I20260807 15:06:16.198361 23770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:16.198266 23769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c0427b040404822ab3f4b22d3fe8b1f. Latest consensus state: current_term: 1 leader_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0427b040404822ab3f4b22d3fe8b1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37243 } } }
I20260807 15:06:16.198418 23769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:16.198776 23775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:16.199352 23775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:06:16.202399 23775 catalog_manager.cc:1347] Generated new cluster ID: 596d3361318443abb7a92c1220c60559
I20260807 15:06:16.202464 23775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:16.204259 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:16.191536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:34721 (local address 127.0.0.1:37243)
0807 15:06:16.191740 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:16.191748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:16.191774 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:06:16.192372 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:16.192382 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:16.192459 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:16.192628 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:16.193623 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:16.194627 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:16.196003 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:16.196676 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:16.196779 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:16.200045 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:16.200050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:16.200050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:16.200065 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:16.202308 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:16.202840 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:16.202843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:16.202843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:16.202958 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:16.203176 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:16.203177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:16.203177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:16.203470 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:16.203616 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:16.203917 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 15:06:16.204026 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260807 15:06:16.205654 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:16.215095 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:16.207185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:50349 (local address 127.0.0.1:37243)
0807 15:06:16.207230 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:16.207232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:16.207403 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:06:16.207491 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:16.207492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:16.207501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:16.207529 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:16.208205 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:16.208984 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:16.210023 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:16.210381 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:16.210423 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:16.212062 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:16.212063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:16.212064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:16.212071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:16.213689 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:16.214131 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:16.214131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:16.214132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:16.214184 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:16.214384 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:16.214384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:16.214385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:16.214511 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:16.214612 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:16.214919 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 15:06:16.214991 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:06:16.216876 23775 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:16.217618 23775 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:16.225088 23775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c0427b040404822ab3f4b22d3fe8b1f: Generated new TSK 0
I20260807 15:06:16.225318 23775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:16.234951 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23694
W20260807 15:06:16.241606 23677 connection.cc:535] client connection to 127.0.0.1:37243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37243): Connection reset by peer (error 104)
2026-08-07T15:06:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23795](info): set up 2 sockets
Aug 07 15:06:16 dist-test-slave-8tdl krb5kdc[23795](info): commencing operation
krb5kdc: starting...
W20260807 15:06:17.335078 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:17 dist-test-slave-8tdl krb5kdc[23795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:17.470649 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:17.562245 23808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:17.562494 23808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:17.562521 23808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:17.563746 23808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:17.563791 23808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:17.563805 23808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:17.563818 23808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:17.563830 23808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:17.565377 23808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:17.565603 23808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:17.565785 23808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:17.571031 23813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:17.571060 23814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:17.571031 23816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:17.571378 23808 server_base.cc:1034] running on GCE node
I20260807 15:06:17.571705 23808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:17.572053 23808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:17.573252 23808 hybrid_clock.cc:648] HybridClock initialized: now 1786115177573231 us; error 35 us; skew 500 ppm
Aug 07 15:06:17 dist-test-slave-8tdl krb5kdc[23795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:17.576170 23808 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:17.576752 23808 webserver.cc:460] Webserver started at http://127.0.0.1:37435/ using document root <none> and password file <none>
I20260807 15:06:17.576998 23808 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:17.577070 23808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:17.577234 23808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:17.578373 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "18cc62cdd5014b5f92f51564c6acfa04"
format_stamp: "Formatted at 2026-08-07 15:06:17 on dist-test-slave-8tdl"
server_key: "7d2ab9fe1f68dbba2b172520b1d1966a"
server_key_iv: "db1ee876d85817ebcde740513acf1018"
server_key_version: "encryptionkey@0"
I20260807 15:06:17.578843 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "18cc62cdd5014b5f92f51564c6acfa04"
format_stamp: "Formatted at 2026-08-07 15:06:17 on dist-test-slave-8tdl"
server_key: "7d2ab9fe1f68dbba2b172520b1d1966a"
server_key_iv: "db1ee876d85817ebcde740513acf1018"
server_key_version: "encryptionkey@0"
I20260807 15:06:17.580332 23808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:17.581351 23824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:17.581614 23808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:17.581730 23808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "18cc62cdd5014b5f92f51564c6acfa04"
format_stamp: "Formatted at 2026-08-07 15:06:17 on dist-test-slave-8tdl"
server_key: "7d2ab9fe1f68dbba2b172520b1d1966a"
server_key_iv: "db1ee876d85817ebcde740513acf1018"
server_key_version: "encryptionkey@0"
I20260807 15:06:17.581828 23808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:17.596328 23808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:17.599025 23808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:17.599216 23808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:17.603775 23808 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34247
I20260807 15:06:17.603816 23878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34247 every 8 connection(s)
I20260807 15:06:17.604205 23808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:17.605167 23879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:17.607365 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23808
I20260807 15:06:17.607502 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:17.607767 1535 external_mini_cluster.cc:1428] Setting key 570093d43542f190013d0f0a9bfbbc40
I20260807 15:06:17.607942 23879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: Bootstrap starting.
I20260807 15:06:17.608728 23879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:17.609058 23879 log.cc:824] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:17.609836 23879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: No bootstrap required, opened a new log
I20260807 15:06:17.611927 23879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260807 15:06:17.612115 23879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:17.612171 23879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cc62cdd5014b5f92f51564c6acfa04, State: Initialized, Role: FOLLOWER
I20260807 15:06:17.612325 23879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260807 15:06:17.612411 23879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:17.612455 23879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:17.612505 23879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:17.613107 23879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260807 15:06:17.613230 23879 leader_election.cc:302] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18cc62cdd5014b5f92f51564c6acfa04; no voters:
I20260807 15:06:17.613438 23879 leader_election.cc:288] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:17.613472 23884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:17.613591 23884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [term 1 LEADER]: Becoming Leader. State: Replica: 18cc62cdd5014b5f92f51564c6acfa04, State: Running, Role: LEADER
I20260807 15:06:17.613729 23879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:17.613729 23884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260807 15:06:17.614104 23884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18cc62cdd5014b5f92f51564c6acfa04. Latest consensus state: current_term: 1 leader_uuid: "18cc62cdd5014b5f92f51564c6acfa04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260807 15:06:17.614116 23885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18cc62cdd5014b5f92f51564c6acfa04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18cc62cdd5014b5f92f51564c6acfa04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260807 15:06:17.614271 23884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:17.614286 23885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:17.614835 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:17.608421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:55083 (local address 127.0.0.1:34247)
0807 15:06:17.608652 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:17.608664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:17.608764 (+ 100us) server_negotiation.cc:407] Connection header received
0807 15:06:17.609546 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:17.609557 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:17.609644 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:17.609815 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:17.610067 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:17.611160 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:17.612060 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:17.612743 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:17.612879 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:17.613784 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:17.613788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:17.613788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:17.614020 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:17.614068 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:17.614085 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:17.614180 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:17.614502 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 15:06:17.614536 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
W20260807 15:06:17.615849 23899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:17.615921 23899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:17.615933 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:17.615998 23891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:17.616185 23891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:17.617971 23891 catalog_manager.cc:1347] Generated new cluster ID: dc6f5a640b26426bb41e2ec25aee6d72
I20260807 15:06:17.618038 23891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:17.621546 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:17.617743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:56535 (local address 127.0.0.1:34247)
0807 15:06:17.617783 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:17.617785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:17.617926 (+ 141us) server_negotiation.cc:407] Connection header received
0807 15:06:17.618017 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:17.618018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:17.618029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:17.618085 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:17.618265 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:17.618862 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:17.619914 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:17.620224 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:17.620299 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:17.621225 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:17.621226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:17.621226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:17.621232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:17.621247 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:17.621252 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:17.621286 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:17.621428 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:06:17.621440 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:17.627734 23891 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:17.628458 23891 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:17.635187 23891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18cc62cdd5014b5f92f51564c6acfa04: Generated new TSK 0
I20260807 15:06:17.635473 23891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:17.643960 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23808
W20260807 15:06:17.652025 23793 connection.cc:535] client connection to 127.0.0.1:34247 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34247): Connection reset by peer (error 104)
2026-08-07T15:06:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:17 dist-test-slave-8tdl krb5kdc[23910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:17 dist-test-slave-8tdl krb5kdc[23910](info): set up 2 sockets
Aug 07 15:06:17 dist-test-slave-8tdl krb5kdc[23910](info): commencing operation
krb5kdc: starting...
W20260807 15:06:18.780309 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:18 dist-test-slave-8tdl krb5kdc[23910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:06:18.927843 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:19.017150 23923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:19.017448 23923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:19.017496 23923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:19.018865 23923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:19.018952 23923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:19.018977 23923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:19.019006 23923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:19.019033 23923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:19.020907 23923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39637
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:19.021239 23923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:19.021507 23923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:19.026952 23928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:19.027057 23923 server_base.cc:1034] running on GCE node
W20260807 15:06:19.027038 23931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:19.027038 23929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:19.027454 23923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:19.027796 23923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:19.029206 23923 hybrid_clock.cc:648] HybridClock initialized: now 1786115179029177 us; error 39 us; skew 500 ppm
Aug 07 15:06:19 dist-test-slave-8tdl krb5kdc[23910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:19.032084 23923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:06:19.032655 23923 webserver.cc:460] Webserver started at http://127.0.0.1:38027/ using document root <none> and password file <none>
I20260807 15:06:19.032946 23923 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:19.033020 23923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:19.033176 23923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:19.034400 23923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "b90f21cb561a4761b74d6e1074d8621a"
format_stamp: "Formatted at 2026-08-07 15:06:19 on dist-test-slave-8tdl"
server_key: "bf52b6da5365ed3b0a1cda08cc345037"
server_key_iv: "452706f05a5c58703ef42ad6d117235f"
server_key_version: "encryptionkey@0"
I20260807 15:06:19.034883 23923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "b90f21cb561a4761b74d6e1074d8621a"
format_stamp: "Formatted at 2026-08-07 15:06:19 on dist-test-slave-8tdl"
server_key: "bf52b6da5365ed3b0a1cda08cc345037"
server_key_iv: "452706f05a5c58703ef42ad6d117235f"
server_key_version: "encryptionkey@0"
I20260807 15:06:19.036399 23923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:19.037338 23938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:19.037563 23923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:19.037698 23923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "b90f21cb561a4761b74d6e1074d8621a"
format_stamp: "Formatted at 2026-08-07 15:06:19 on dist-test-slave-8tdl"
server_key: "bf52b6da5365ed3b0a1cda08cc345037"
server_key_iv: "452706f05a5c58703ef42ad6d117235f"
server_key_version: "encryptionkey@0"
I20260807 15:06:19.037786 23923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:19.041512 23923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:19.044067 23923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:19.044250 23923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:19.048774 23923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39637
I20260807 15:06:19.048810 23991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39637 every 8 connection(s)
I20260807 15:06:19.049187 23923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:19.050341 23992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:19.052811 23992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a: Bootstrap starting.
I20260807 15:06:19.053409 23992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:19.053668 23992 log.cc:824] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:19.054467 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 23923
I20260807 15:06:19.054538 23992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a: No bootstrap required, opened a new log
I20260807 15:06:19.054621 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:19.054849 1535 external_mini_cluster.cc:1428] Setting key 95789cf0794fc7112036f022e61e7a1d
I20260807 15:06:19.057404 23992 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260807 15:06:19.057626 23992 raft_consensus.cc:374] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:19.057693 23992 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90f21cb561a4761b74d6e1074d8621a, State: Initialized, Role: FOLLOWER
I20260807 15:06:19.057832 23992 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260807 15:06:19.057933 23992 raft_consensus.cc:388] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:19.057968 23992 raft_consensus.cc:482] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:19.058077 23992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:19.058288 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:19.055443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:39101 (local address 127.0.0.1:39637)
0807 15:06:19.055676 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:19.055686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:19.055722 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:06:19.056322 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:19.056332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:19.056368 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:19.056552 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:19.057666 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:19.057671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:19.057671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:19.057677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:19.057749 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:19.057766 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:19.057971 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:19.058040 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 15:06:19.058092 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":119,"threads_started":1}
I20260807 15:06:19.059238 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:19.059273 23992 raft_consensus.cc:504] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260807 15:06:19.059398 23992 leader_election.cc:302] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b90f21cb561a4761b74d6e1074d8621a; no voters:
I20260807 15:06:19.059594 23992 leader_election.cc:288] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:19.059625 23998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:19.059742 23998 raft_consensus.cc:686] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [term 1 LEADER]: Becoming Leader. State: Replica: b90f21cb561a4761b74d6e1074d8621a, State: Running, Role: LEADER
I20260807 15:06:19.059880 23992 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:19.059894 23998 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260807 15:06:19.060371 23999 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b90f21cb561a4761b74d6e1074d8621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260807 15:06:19.060654 23999 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:19.060572 24000 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90f21cb561a4761b74d6e1074d8621a. Latest consensus state: current_term: 1 leader_uuid: "b90f21cb561a4761b74d6e1074d8621a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90f21cb561a4761b74d6e1074d8621a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260807 15:06:19.060961 24000 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:19.060977 24011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:19.061573 24011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:19.063256 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:19.061737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.51:38745 (local address 127.0.0.1:39637)
0807 15:06:19.061779 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:19.061781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:19.062071 (+ 290us) server_negotiation.cc:407] Connection header received
0807 15:06:19.062197 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:19.062198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:19.062203 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:19.062244 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:19.063042 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:19.063043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:19.063043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:19.063048 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:19.063062 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:19.063066 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:19.063111 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:19.063175 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:06:19.063190 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:19.063398 24011 catalog_manager.cc:1347] Generated new cluster ID: defaa48d0af14cb9889484c7187e2ece
I20260807 15:06:19.063449 24011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:19.075716 24011 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:19.075994 24011 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:19.081622 24011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b90f21cb561a4761b74d6e1074d8621a: Generated new TSK 0
I20260807 15:06:19.081843 24011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:19.084686 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 23923
2026-08-07T15:06:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:19 dist-test-slave-8tdl krb5kdc[24025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:19 dist-test-slave-8tdl krb5kdc[24025](info): set up 2 sockets
Aug 07 15:06:19 dist-test-slave-8tdl krb5kdc[24025](info): commencing operation
krb5kdc: starting...
W20260807 15:06:20.184819 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:20.334422 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40097
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:40097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:20.422282 24038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:20.422574 24038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:20.422622 24038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:20.423979 24038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:20.424074 24038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:20.424101 24038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:20.424135 24038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:20.424183 24038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:20.426337 24038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:40097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:40097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:20.426795 24038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:20.427095 24038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:20.432781 24038 server_base.cc:1034] running on GCE node
W20260807 15:06:20.432752 24044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:20.432741 24047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:20.432726 24045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:20.433283 24038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:20.433598 24038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:20.434798 24038 hybrid_clock.cc:648] HybridClock initialized: now 1786115180434774 us; error 38 us; skew 500 ppm
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115180, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:20.442983 24038 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:20.443567 24038 webserver.cc:460] Webserver started at http://127.1.127.254:38799/ using document root <none> and password file <none>
I20260807 15:06:20.443799 24038 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:20.443842 24038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:20.443941 24038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:20.445039 24038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b"
format_stamp: "Formatted at 2026-08-07 15:06:20 on dist-test-slave-8tdl"
server_key: "b63082553d4c81bdabcf437654a1762b"
server_key_iv: "aab0487a986b1be77789dfb5773811d9"
server_key_version: "encryptionkey@0"
I20260807 15:06:20.445361 24038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b"
format_stamp: "Formatted at 2026-08-07 15:06:20 on dist-test-slave-8tdl"
server_key: "b63082553d4c81bdabcf437654a1762b"
server_key_iv: "aab0487a986b1be77789dfb5773811d9"
server_key_version: "encryptionkey@0"
I20260807 15:06:20.446851 24038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:20.447721 24054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:20.448169 24038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:20.448267 24038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b"
format_stamp: "Formatted at 2026-08-07 15:06:20 on dist-test-slave-8tdl"
server_key: "b63082553d4c81bdabcf437654a1762b"
server_key_iv: "aab0487a986b1be77789dfb5773811d9"
server_key_version: "encryptionkey@0"
I20260807 15:06:20.448382 24038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:20.464280 24038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:20.467200 24038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:20.467384 24038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:20.473153 24038 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:40097
I20260807 15:06:20.473260 24107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:40097 every 8 connection(s)
I20260807 15:06:20.473673 24038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:20.474743 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:20.477699 24108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: Bootstrap starting.
I20260807 15:06:20.478480 24108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:20.478864 24108 log.cc:824] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:20.479794 24108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: No bootstrap required, opened a new log
I20260807 15:06:20.481251 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24038
I20260807 15:06:20.481365 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:20.481680 1535 external_mini_cluster.cc:1428] Setting key 9c1aa87f1766ab9781e5695c7e8b5c01
I20260807 15:06:20.481900 24108 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } }
I20260807 15:06:20.482090 24108 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:20.482161 24108 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f41e1e7aee4258bac01d26aa7c5f3b, State: Initialized, Role: FOLLOWER
I20260807 15:06:20.482374 24108 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } }
I20260807 15:06:20.482543 24108 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:20.482610 24108 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:20.482692 24108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:20.483613 24108 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } }
I20260807 15:06:20.483738 24108 leader_election.cc:302] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f41e1e7aee4258bac01d26aa7c5f3b; no voters:
I20260807 15:06:20.483920 24108 leader_election.cc:288] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:20.483950 24113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:20.484076 24113 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [term 1 LEADER]: Becoming Leader. State: Replica: e1f41e1e7aee4258bac01d26aa7c5f3b, State: Running, Role: LEADER
I20260807 15:06:20.484225 24108 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:20.484217 24113 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } }
I20260807 15:06:20.484735 24116 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f41e1e7aee4258bac01d26aa7c5f3b. Latest consensus state: current_term: 1 leader_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } } }
I20260807 15:06:20.484840 24116 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:20.485071 24115 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f41e1e7aee4258bac01d26aa7c5f3b" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 40097 } } }
I20260807 15:06:20.485428 24115 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b [sys.catalog]: This master's current role is: LEADER
W20260807 15:06:20.486287 24130 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:20.486379 24130 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:20.486434 24121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:20.486619 24121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:20.488549 24121 catalog_manager.cc:1347] Generated new cluster ID: 502869971b924410a4ff32c01d6d4dd8
I20260807 15:06:20.488613 24121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:20.494400 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:20.482339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60967 (local address 127.1.127.254:40097)
0807 15:06:20.482542 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:20.482555 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:20.482612 (+ 57us) server_negotiation.cc:407] Connection header received
0807 15:06:20.482783 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:20.482791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:20.482854 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:20.482969 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:20.483832 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:20.485506 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:20.486697 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:20.487158 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:20.487290 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:20.490057 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:20.490062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:20.490062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:20.490075 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:20.492105 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:20.492562 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:20.492565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:20.492565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:20.493186 (+ 621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:20.493460 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:20.493461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:20.493461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:20.493763 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:20.493974 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:20.494108 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:06:20.494234 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260807 15:06:20.495504 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:20.506067 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:20.499189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48019 (local address 127.1.127.254:40097)
0807 15:06:20.499248 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:20.499250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:20.499498 (+ 248us) server_negotiation.cc:407] Connection header received
0807 15:06:20.499579 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:20.499580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:20.499591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:20.499643 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:20.500213 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:20.500735 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:20.501485 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:20.501729 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:20.501821 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:20.503156 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:20.503157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:20.503157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:20.503165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:20.504699 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:20.505099 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:20.505100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:20.505101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:20.505166 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:20.505418 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:20.505419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:20.505419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:20.505540 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:20.505677 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:20.505854 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:06:20.505941 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:06:20.508713 24121 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:20.509353 24121 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:20.513135 24121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f41e1e7aee4258bac01d26aa7c5f3b: Generated new TSK 0
I20260807 15:06:20.513355 24121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:20.516513 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24038
W20260807 15:06:20.524055 24023 connection.cc:535] client connection to 127.1.127.254:40097 recv error: Network error: failed to read from TLS socket (remote: 127.1.127.254:40097): Connection reset by peer (error 104)
2026-08-07T15:06:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24141](info): set up 2 sockets
Aug 07 15:06:20 dist-test-slave-8tdl krb5kdc[24141](info): commencing operation
krb5kdc: starting...
W20260807 15:06:21.612702 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:21 dist-test-slave-8tdl krb5kdc[24141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:21.754274 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37031
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:21.844679 24154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:21.844955 24154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:21.845021 24154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:21.846344 24154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:21.846427 24154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:21.846455 24154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:21.846479 24154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:21.846506 24154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:21.848218 24154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:21.848524 24154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:21.848783 24154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:21.854377 24154 server_base.cc:1034] running on GCE node
W20260807 15:06:21.854336 24160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:21.854342 24162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:21.854301 24159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:21.855029 24154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:21.855435 24154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:21.856695 24154 hybrid_clock.cc:648] HybridClock initialized: now 1786115181856676 us; error 42 us; skew 500 ppm
Aug 07 15:06:21 dist-test-slave-8tdl krb5kdc[24141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:21.860121 24154 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:21.860757 24154 webserver.cc:460] Webserver started at http://127.1.127.254:44917/ using document root <none> and password file <none>
I20260807 15:06:21.861061 24154 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:21.861142 24154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:21.861320 24154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:21.862619 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "bfc03277cd7847ebad9eecbdad37b668"
format_stamp: "Formatted at 2026-08-07 15:06:21 on dist-test-slave-8tdl"
server_key: "e110be2101915ef8e2ca7495d7f6d285"
server_key_iv: "8667170ca78593484e5f53e84f492a2f"
server_key_version: "encryptionkey@0"
I20260807 15:06:21.863133 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "bfc03277cd7847ebad9eecbdad37b668"
format_stamp: "Formatted at 2026-08-07 15:06:21 on dist-test-slave-8tdl"
server_key: "e110be2101915ef8e2ca7495d7f6d285"
server_key_iv: "8667170ca78593484e5f53e84f492a2f"
server_key_version: "encryptionkey@0"
I20260807 15:06:21.865005 24154 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:21.866019 24169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:21.866245 24154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:21.866367 24154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "bfc03277cd7847ebad9eecbdad37b668"
format_stamp: "Formatted at 2026-08-07 15:06:21 on dist-test-slave-8tdl"
server_key: "e110be2101915ef8e2ca7495d7f6d285"
server_key_iv: "8667170ca78593484e5f53e84f492a2f"
server_key_version: "encryptionkey@0"
I20260807 15:06:21.866467 24154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:21.877231 24154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:21.880023 24154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:21.880223 24154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:21.885356 24154 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:37031
I20260807 15:06:21.885526 24223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:37031 every 8 connection(s)
I20260807 15:06:21.885891 24154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:21.887115 24224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:21.889647 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668: Bootstrap starting.
I20260807 15:06:21.890319 24224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:21.890712 24224 log.cc:824] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:21.890905 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24154
I20260807 15:06:21.891062 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:21.891297 1535 external_mini_cluster.cc:1428] Setting key cb3a940b2bbb74d2c8e05ebffddcf8af
I20260807 15:06:21.891778 24224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668: No bootstrap required, opened a new log
I20260807 15:06:21.893651 24224 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } }
I20260807 15:06:21.893798 24224 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:21.893821 24224 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc03277cd7847ebad9eecbdad37b668, State: Initialized, Role: FOLLOWER
I20260807 15:06:21.893977 24224 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } }
I20260807 15:06:21.894071 24224 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:21.894109 24224 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:21.894157 24224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:21.895040 24224 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } }
I20260807 15:06:21.895161 24224 leader_election.cc:302] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc03277cd7847ebad9eecbdad37b668; no voters:
I20260807 15:06:21.895363 24224 leader_election.cc:288] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:21.895429 24229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:21.895558 24229 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [term 1 LEADER]: Becoming Leader. State: Replica: bfc03277cd7847ebad9eecbdad37b668, State: Running, Role: LEADER
I20260807 15:06:21.895658 24224 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:21.895642 24229 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } }
I20260807 15:06:21.896008 24229 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc03277cd7847ebad9eecbdad37b668. Latest consensus state: current_term: 1 leader_uuid: "bfc03277cd7847ebad9eecbdad37b668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } } }
I20260807 15:06:21.896101 24229 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:21.896021 24230 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc03277cd7847ebad9eecbdad37b668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc03277cd7847ebad9eecbdad37b668" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37031 } } }
I20260807 15:06:21.896266 24230 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:21.896351 24235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:21.896920 24235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:21.898923 24235 catalog_manager.cc:1347] Generated new cluster ID: 6ab4741971d74d30a697d30001966685
I20260807 15:06:21.898981 24235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:21.899873 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:21.891911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49111 (local address 127.1.127.254:37031)
0807 15:06:21.892130 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:21.892139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:21.892165 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:06:21.893666 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:21.893675 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:21.893738 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:21.894086 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:21.894314 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:21.896198 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:21.897450 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:21.898087 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:21.898570 (+ 483us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:21.899075 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:21.899079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:21.899079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:21.899313 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:21.899352 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:21.899368 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:21.899462 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:21.899592 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 15:06:21.899623 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260807 15:06:21.900811 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:21.910604 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:21.902990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50931 (local address 127.1.127.254:37031)
0807 15:06:21.903032 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:21.903034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:21.906919 (+ 3885us) server_negotiation.cc:407] Connection header received
0807 15:06:21.906988 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:21.906990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:21.907004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:21.907063 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:21.907229 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:21.908027 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:21.909072 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:21.909446 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:21.909523 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:21.910315 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:21.910316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:21.910316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:21.910322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:21.910340 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:21.910345 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:21.910413 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:21.910491 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:06:21.910505 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:21.915306 24235 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:21.916107 24235 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:21.926638 24235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc03277cd7847ebad9eecbdad37b668: Generated new TSK 0
I20260807 15:06:21.926934 24235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:21.935112 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24154
W20260807 15:06:21.942374 24139 connection.cc:535] client connection to 127.1.127.254:37031 recv error: Network error: failed to read from TLS socket (remote: 127.1.127.254:37031): Connection reset by peer (error 104)
2026-08-07T15:06:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:22 dist-test-slave-8tdl krb5kdc[24255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:22 dist-test-slave-8tdl krb5kdc[24255](info): set up 2 sockets
Aug 07 15:06:22 dist-test-slave-8tdl krb5kdc[24255](info): commencing operation
krb5kdc: starting...
W20260807 15:06:23.035713 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:23 dist-test-slave-8tdl krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:23.175518 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33719
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:33719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:23.260143 24268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:23.260425 24268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:23.260473 24268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:23.261752 24268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:23.261835 24268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:23.261861 24268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:23.261893 24268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:23.261951 24268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:23.263597 24268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:33719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:33719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:23.263921 24268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:23.264180 24268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:23.269503 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:23.269503 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:23.269582 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:23.269989 24268 server_base.cc:1034] running on GCE node
I20260807 15:06:23.270260 24268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:23.270601 24268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:23.271785 24268 hybrid_clock.cc:648] HybridClock initialized: now 1786115183271760 us; error 41 us; skew 500 ppm
Aug 07 15:06:23 dist-test-slave-8tdl krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115183, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:23.274561 24268 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:23.275125 24268 webserver.cc:460] Webserver started at http://127.1.127.254:41543/ using document root <none> and password file <none>
I20260807 15:06:23.275367 24268 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:23.275413 24268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:23.275576 24268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:23.276811 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "e3aad416e24044a2b0d239285e8be74e"
format_stamp: "Formatted at 2026-08-07 15:06:23 on dist-test-slave-8tdl"
server_key: "e381833778925ea51b51fb864913bfa5"
server_key_iv: "32ed6b2b1ec88434e74a375bcf9faf53"
server_key_version: "encryptionkey@0"
I20260807 15:06:23.277223 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "e3aad416e24044a2b0d239285e8be74e"
format_stamp: "Formatted at 2026-08-07 15:06:23 on dist-test-slave-8tdl"
server_key: "e381833778925ea51b51fb864913bfa5"
server_key_iv: "32ed6b2b1ec88434e74a375bcf9faf53"
server_key_version: "encryptionkey@0"
I20260807 15:06:23.278862 24268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:23.279930 24283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:23.280277 24268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:23.280433 24268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "e3aad416e24044a2b0d239285e8be74e"
format_stamp: "Formatted at 2026-08-07 15:06:23 on dist-test-slave-8tdl"
server_key: "e381833778925ea51b51fb864913bfa5"
server_key_iv: "32ed6b2b1ec88434e74a375bcf9faf53"
server_key_version: "encryptionkey@0"
I20260807 15:06:23.280536 24268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:23.284416 24268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:23.286967 24268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:23.287137 24268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:23.291899 24268 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:33719
I20260807 15:06:23.291958 24337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:33719 every 8 connection(s)
I20260807 15:06:23.292343 24268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:23.293531 24338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:23.296372 24338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e: Bootstrap starting.
I20260807 15:06:23.297801 24338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:23.298239 24338 log.cc:824] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:23.299386 24338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e: No bootstrap required, opened a new log
I20260807 15:06:23.301488 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24268
I20260807 15:06:23.301445 24338 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } }
I20260807 15:06:23.301602 24338 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:23.301656 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:23.301679 24338 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3aad416e24044a2b0d239285e8be74e, State: Initialized, Role: FOLLOWER
I20260807 15:06:23.301887 24338 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } }
I20260807 15:06:23.301963 24338 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:23.302012 24338 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:23.302050 1535 external_mini_cluster.cc:1428] Setting key c9aba91d52b8748f317bd1ac6339958f
I20260807 15:06:23.302069 24338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:23.303596 24338 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } }
I20260807 15:06:23.303742 24338 leader_election.cc:302] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3aad416e24044a2b0d239285e8be74e; no voters:
I20260807 15:06:23.303941 24338 leader_election.cc:288] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:23.304028 24343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:23.304196 24343 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [term 1 LEADER]: Becoming Leader. State: Replica: e3aad416e24044a2b0d239285e8be74e, State: Running, Role: LEADER
I20260807 15:06:23.304278 24338 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:23.304304 24343 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } }
I20260807 15:06:23.304706 24345 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3aad416e24044a2b0d239285e8be74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } } }
I20260807 15:06:23.304703 24343 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3aad416e24044a2b0d239285e8be74e. Latest consensus state: current_term: 1 leader_uuid: "e3aad416e24044a2b0d239285e8be74e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3aad416e24044a2b0d239285e8be74e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 33719 } } }
I20260807 15:06:23.304829 24345 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:23.304837 24343 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:23.305451 24349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:23.305490 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:23.302772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48455 (local address 127.1.127.254:33719)
0807 15:06:23.303075 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:23.303090 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:23.303140 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:06:23.303296 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:23.303304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:23.303341 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:23.303516 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:23.304313 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:23.304317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:23.304318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:23.304324 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:23.304379 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:23.304396 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:23.304721 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:23.304757 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 15:06:23.304811 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":157,"threads_started":1}
I20260807 15:06:23.306113 24349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:23.306742 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:23.309912 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:23.308435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36447 (local address 127.1.127.254:33719)
0807 15:06:23.308475 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:23.308477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:23.308792 (+ 315us) server_negotiation.cc:407] Connection header received
0807 15:06:23.308821 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:23.308822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:23.308828 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:23.308889 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:23.309566 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:23.309567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:23.309567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:23.309572 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:23.309591 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:23.309595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:23.309650 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:23.309773 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 15:06:23.309786 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:06:23.311199 24349 catalog_manager.cc:1347] Generated new cluster ID: 3aed82979f464030bbc8b6a9d160ef05
I20260807 15:06:23.311272 24349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:23.328277 24349 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:23.328512 24349 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:23.339164 24349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3aad416e24044a2b0d239285e8be74e: Generated new TSK 0
I20260807 15:06:23.339427 24349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:23.342124 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24268
2026-08-07T15:06:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:23 dist-test-slave-8tdl krb5kdc[24369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:23 dist-test-slave-8tdl krb5kdc[24369](info): set up 2 sockets
Aug 07 15:06:23 dist-test-slave-8tdl krb5kdc[24369](info): commencing operation
krb5kdc: starting...
W20260807 15:06:24.465971 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:24.606380 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46409
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:24.693830 24383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:24.694063 24383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:24.694089 24383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:24.695304 24383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:24.695353 24383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:24.695367 24383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:24.695381 24383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:24.695393 24383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:24.696990 24383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:46409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:24.697234 24383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:24.697436 24383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:24.702873 24389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:24.702951 24388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:24.703074 24383 server_base.cc:1034] running on GCE node
W20260807 15:06:24.702848 24391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:24.703433 24383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:24.703758 24383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:24.704977 24383 hybrid_clock.cc:648] HybridClock initialized: now 1786115184704995 us; error 106 us; skew 500 ppm
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115184, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:24.708161 24383 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:24.708683 24383 webserver.cc:460] Webserver started at http://127.1.127.254:38703/ using document root <none> and password file <none>
I20260807 15:06:24.708940 24383 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:24.709017 24383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:24.709169 24383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:24.710289 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "b041ddc69c754eb48b73c733c25e8f1e"
format_stamp: "Formatted at 2026-08-07 15:06:24 on dist-test-slave-8tdl"
server_key: "ccd7116d9c521e85752d497c81f80708"
server_key_iv: "c572fea85ce2dd0d9fa4f9a2c0a1a6b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:24.710664 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "b041ddc69c754eb48b73c733c25e8f1e"
format_stamp: "Formatted at 2026-08-07 15:06:24 on dist-test-slave-8tdl"
server_key: "ccd7116d9c521e85752d497c81f80708"
server_key_iv: "c572fea85ce2dd0d9fa4f9a2c0a1a6b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:24.712164 24383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:24.713003 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:24.713335 24383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:24.713449 24383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "b041ddc69c754eb48b73c733c25e8f1e"
format_stamp: "Formatted at 2026-08-07 15:06:24 on dist-test-slave-8tdl"
server_key: "ccd7116d9c521e85752d497c81f80708"
server_key_iv: "c572fea85ce2dd0d9fa4f9a2c0a1a6b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:24.713539 24383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:24.731108 24383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:24.733534 24383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:24.733718 24383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:24.738531 24383 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:46409
I20260807 15:06:24.738571 24451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:46409 every 8 connection(s)
I20260807 15:06:24.738996 24383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:24.739899 24452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:24.742264 24452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e: Bootstrap starting.
I20260807 15:06:24.742895 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24383
I20260807 15:06:24.742990 24452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:24.743005 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:24.743211 1535 external_mini_cluster.cc:1428] Setting key e6fd3b47b67834af5f076356abd22d22
I20260807 15:06:24.743366 24452 log.cc:824] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:24.744405 24452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e: No bootstrap required, opened a new log
I20260807 15:06:24.747006 24452 raft_consensus.cc:348] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } }
I20260807 15:06:24.747197 24452 raft_consensus.cc:374] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:24.747228 24452 raft_consensus.cc:729] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b041ddc69c754eb48b73c733c25e8f1e, State: Initialized, Role: FOLLOWER
I20260807 15:06:24.747356 24452 consensus_queue.cc:260] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } }
I20260807 15:06:24.747463 24452 raft_consensus.cc:388] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:24.747538 24452 raft_consensus.cc:482] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:24.747598 24452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:24.748263 24452 raft_consensus.cc:504] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } }
I20260807 15:06:24.748373 24452 leader_election.cc:302] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b041ddc69c754eb48b73c733c25e8f1e; no voters:
I20260807 15:06:24.748553 24452 leader_election.cc:288] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:24.748623 24457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:24.748799 24457 raft_consensus.cc:686] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [term 1 LEADER]: Becoming Leader. State: Replica: b041ddc69c754eb48b73c733c25e8f1e, State: Running, Role: LEADER
I20260807 15:06:24.748807 24452 sys_catalog.cc:564] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:24.748929 24457 consensus_queue.cc:237] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } }
I20260807 15:06:24.749294 24458 sys_catalog.cc:455] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b041ddc69c754eb48b73c733c25e8f1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } } }
I20260807 15:06:24.749284 24457 sys_catalog.cc:455] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b041ddc69c754eb48b73c733c25e8f1e. Latest consensus state: current_term: 1 leader_uuid: "b041ddc69c754eb48b73c733c25e8f1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b041ddc69c754eb48b73c733c25e8f1e" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 46409 } } }
I20260807 15:06:24.749382 24458 sys_catalog.cc:458] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:24.749387 24457 sys_catalog.cc:458] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:24.749579 24462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:24.750492 24462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:24.752499 24462 catalog_manager.cc:1347] Generated new cluster ID: cee16a5ca7064603956182ad09771470
I20260807 15:06:24.752576 24462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:24.756245 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:24.743843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39187 (local address 127.1.127.254:46409)
0807 15:06:24.744019 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:24.744031 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:24.744110 (+ 79us) server_negotiation.cc:407] Connection header received
0807 15:06:24.744764 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:24.744773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:24.744838 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:24.745118 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:24.745785 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:24.747305 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:24.748243 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:24.749180 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:24.749793 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:24.752387 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:24.752391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:24.752392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:24.752403 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:24.754470 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:24.754921 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:24.754925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:24.754925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:24.755053 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:24.755263 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:24.755264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:24.755264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:24.755595 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:24.755761 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:24.755856 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:06:24.756018 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260807 15:06:24.757370 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:24.758890 24462 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:24.759582 24462 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:24.766315 24462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b041ddc69c754eb48b73c733c25e8f1e: Generated new TSK 0
I20260807 15:06:24.766533 24462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:24.767303 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:24.759075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43255 (local address 127.1.127.254:46409)
0807 15:06:24.759130 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:24.759132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:24.759375 (+ 243us) server_negotiation.cc:407] Connection header received
0807 15:06:24.759407 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:24.759408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:24.759603 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:24.759677 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:24.760442 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:24.761186 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:24.762283 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:24.762622 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:24.762706 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:24.764176 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:24.764177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:24.764177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:24.764185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:24.765840 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:24.766300 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:24.766301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:24.766302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:24.766366 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:24.766763 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:24.766764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:24.766764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:24.766890 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:24.767009 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:24.767117 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:06:24.767207 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:24.769218 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24383
2026-08-07T15:06:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24484](info): set up 2 sockets
Aug 07 15:06:24 dist-test-slave-8tdl krb5kdc[24484](info): commencing operation
krb5kdc: starting...
W20260807 15:06:25.858318 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:25 dist-test-slave-8tdl krb5kdc[24484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:25.999261 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41527
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:26.087419 24497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:26.087656 24497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:26.087682 24497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:26.088862 24497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:26.088912 24497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:26.088928 24497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:26.088941 24497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:26.088953 24497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:26.090514 24497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:26.090834 24497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:26.091055 24497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:26.096611 24503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:26.096652 24502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:26.096661 24505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:26.096823 24497 server_base.cc:1034] running on GCE node
I20260807 15:06:26.097214 24497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:26.097544 24497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:26.098775 24497 hybrid_clock.cc:648] HybridClock initialized: now 1786115186098721 us; error 76 us; skew 500 ppm
Aug 07 15:06:26 dist-test-slave-8tdl krb5kdc[24484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115186, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:26.107618 24497 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:26.108170 24497 webserver.cc:460] Webserver started at http://127.1.127.254:36461/ using document root <none> and password file <none>
I20260807 15:06:26.108428 24497 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:26.108479 24497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:26.108614 24497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:26.109724 24497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "47d6d3c212714984a9ea91ff1a38af72"
format_stamp: "Formatted at 2026-08-07 15:06:26 on dist-test-slave-8tdl"
server_key: "c5ebe75eb17af5ee99f3bca3ca5d0e8d"
server_key_iv: "6d1d3676c82f1b83b925232b7df197fa"
server_key_version: "encryptionkey@0"
I20260807 15:06:26.110122 24497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "47d6d3c212714984a9ea91ff1a38af72"
format_stamp: "Formatted at 2026-08-07 15:06:26 on dist-test-slave-8tdl"
server_key: "c5ebe75eb17af5ee99f3bca3ca5d0e8d"
server_key_iv: "6d1d3676c82f1b83b925232b7df197fa"
server_key_version: "encryptionkey@0"
I20260807 15:06:26.111775 24497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:26.112867 24512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:26.113222 24497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:26.113332 24497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "47d6d3c212714984a9ea91ff1a38af72"
format_stamp: "Formatted at 2026-08-07 15:06:26 on dist-test-slave-8tdl"
server_key: "c5ebe75eb17af5ee99f3bca3ca5d0e8d"
server_key_iv: "6d1d3676c82f1b83b925232b7df197fa"
server_key_version: "encryptionkey@0"
I20260807 15:06:26.113397 24497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:26.132023 24497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:26.134490 24497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:26.134613 24497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:26.138928 24497 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:41527
I20260807 15:06:26.139030 24565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:41527 every 8 connection(s)
I20260807 15:06:26.139423 24497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:26.140523 24566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:26.143101 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72: Bootstrap starting.
I20260807 15:06:26.143728 24566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:26.144049 24566 log.cc:824] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:26.144868 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72: No bootstrap required, opened a new log
I20260807 15:06:26.146768 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24497
I20260807 15:06:26.146906 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:26.146874 24566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } }
I20260807 15:06:26.147097 1535 external_mini_cluster.cc:1428] Setting key efc1cd749b50dfc4b3d99689e07724a7
I20260807 15:06:26.147086 24566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:26.147131 24566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d6d3c212714984a9ea91ff1a38af72, State: Initialized, Role: FOLLOWER
I20260807 15:06:26.147330 24566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } }
I20260807 15:06:26.147429 24566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:26.147469 24566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:26.147529 24566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:26.148775 24566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } }
I20260807 15:06:26.148890 24566 leader_election.cc:302] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47d6d3c212714984a9ea91ff1a38af72; no voters:
I20260807 15:06:26.149076 24566 leader_election.cc:288] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:26.149257 24571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:26.149395 24566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:26.149485 24571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [term 1 LEADER]: Becoming Leader. State: Replica: 47d6d3c212714984a9ea91ff1a38af72, State: Running, Role: LEADER
I20260807 15:06:26.149600 24571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } }
I20260807 15:06:26.149992 24571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47d6d3c212714984a9ea91ff1a38af72. Latest consensus state: current_term: 1 leader_uuid: "47d6d3c212714984a9ea91ff1a38af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } } }
I20260807 15:06:26.150102 24571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:26.150833 24581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:26.150003 24572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47d6d3c212714984a9ea91ff1a38af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d6d3c212714984a9ea91ff1a38af72" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 41527 } } }
I20260807 15:06:26.151281 24572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:26.151645 24581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:26.153359 24581 catalog_manager.cc:1347] Generated new cluster ID: 8c3319a6b2f74b108465b3b85d352bcd
I20260807 15:06:26.153415 24581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:26.153939 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:26.147652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48609 (local address 127.1.127.254:41527)
0807 15:06:26.147884 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:26.147894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:26.147935 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:06:26.148077 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:26.148086 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:26.148156 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:26.148345 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:26.148624 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:26.150024 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:26.151343 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:26.152101 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:26.152253 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:26.153157 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:26.153161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:26.153161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:26.153397 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:26.153440 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:26.153458 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:26.153539 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:26.153622 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:06:26.153697 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260807 15:06:26.155016 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:26.160504 24581 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:26.161162 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:26.156775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54127 (local address 127.1.127.254:41527)
0807 15:06:26.156818 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:26.156820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:26.156913 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:06:26.157047 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:26.157048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:26.157061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:26.157095 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:26.157350 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:26.158220 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:26.159485 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:26.159837 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:26.159886 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:26.160807 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:26.160809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:26.160810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:26.160843 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:26.160859 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:26.160864 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:26.160899 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:26.161025 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:06:26.161060 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":23}
I20260807 15:06:26.161640 24581 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:26.163935 24581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47d6d3c212714984a9ea91ff1a38af72: Generated new TSK 0
I20260807 15:06:26.164108 24581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:26.170806 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24497
2026-08-07T15:06:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:26 dist-test-slave-8tdl krb5kdc[24598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:26 dist-test-slave-8tdl krb5kdc[24598](info): set up 2 sockets
Aug 07 15:06:26 dist-test-slave-8tdl krb5kdc[24598](info): commencing operation
krb5kdc: starting...
W20260807 15:06:27.267627 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:27 dist-test-slave-8tdl krb5kdc[24598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:27.409569 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36133
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:36133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:27.498292 24612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:27.498584 24612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:27.498625 24612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:27.500000 24612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:27.500092 24612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:27.500115 24612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:27.500136 24612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:27.500167 24612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:27.501822 24612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:36133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:36133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:27.502140 24612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:27.502399 24612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:27.508257 24618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:27.508252 24621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:27.508257 24617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:27.508687 24612 server_base.cc:1034] running on GCE node
I20260807 15:06:27.508976 24612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:27.509352 24612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:27.510545 24612 hybrid_clock.cc:648] HybridClock initialized: now 1786115187510517 us; error 48 us; skew 500 ppm
Aug 07 15:06:27 dist-test-slave-8tdl krb5kdc[24598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115187, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:27.513381 24612 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:27.513880 24612 webserver.cc:460] Webserver started at http://127.1.127.254:41991/ using document root <none> and password file <none>
I20260807 15:06:27.514074 24612 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:27.514115 24612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:27.514210 24612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:27.515389 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f"
format_stamp: "Formatted at 2026-08-07 15:06:27 on dist-test-slave-8tdl"
server_key: "dbdc4d6d57675659e731de30d3760e48"
server_key_iv: "f299b7ac36916b4d68d769ceae755eef"
server_key_version: "encryptionkey@0"
I20260807 15:06:27.515702 24612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f"
format_stamp: "Formatted at 2026-08-07 15:06:27 on dist-test-slave-8tdl"
server_key: "dbdc4d6d57675659e731de30d3760e48"
server_key_iv: "f299b7ac36916b4d68d769ceae755eef"
server_key_version: "encryptionkey@0"
I20260807 15:06:27.517277 24612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:27.518138 24628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:27.518394 24612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:27.518473 24612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f"
format_stamp: "Formatted at 2026-08-07 15:06:27 on dist-test-slave-8tdl"
server_key: "dbdc4d6d57675659e731de30d3760e48"
server_key_iv: "f299b7ac36916b4d68d769ceae755eef"
server_key_version: "encryptionkey@0"
I20260807 15:06:27.518569 24612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:27.522104 24612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:27.524581 24612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:27.524742 24612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:27.529299 24612 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:36133
I20260807 15:06:27.529367 24681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:36133 every 8 connection(s)
I20260807 15:06:27.529732 24612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:27.530936 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:27.533471 24682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: Bootstrap starting.
I20260807 15:06:27.534112 24682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:27.534435 24682 log.cc:824] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:27.535416 24682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: No bootstrap required, opened a new log
I20260807 15:06:27.536296 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24612
I20260807 15:06:27.536396 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:27.536635 1535 external_mini_cluster.cc:1428] Setting key f1f667477d4d7c73cd1bf41af95c2462
I20260807 15:06:27.537952 24682 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } }
I20260807 15:06:27.538142 24682 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:27.538183 24682 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d356fac1684dc2a4c1d98b1cc2ad7f, State: Initialized, Role: FOLLOWER
I20260807 15:06:27.538355 24682 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } }
I20260807 15:06:27.538460 24682 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:27.538507 24682 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:27.538558 24682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:27.539543 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:27.537257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59855 (local address 127.1.127.254:36133)
0807 15:06:27.537491 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:27.537501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:27.537540 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:06:27.537900 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:27.537910 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:27.537946 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:27.538107 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:27.538995 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:27.538998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:27.538999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:27.539003 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:27.539053 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:27.539066 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:27.539128 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:27.539250 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:06:27.539316 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260807 15:06:27.539881 24682 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } }
I20260807 15:06:27.539999 24682 leader_election.cc:302] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7d356fac1684dc2a4c1d98b1cc2ad7f; no voters:
I20260807 15:06:27.540201 24682 leader_election.cc:288] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:27.540225 24687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:27.540400 24687 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [term 1 LEADER]: Becoming Leader. State: Replica: a7d356fac1684dc2a4c1d98b1cc2ad7f, State: Running, Role: LEADER
I20260807 15:06:27.540498 24687 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } }
I20260807 15:06:27.540608 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:27.540479 24682 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:27.541857 24688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } } }
I20260807 15:06:27.541961 24688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:27.542232 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7d356fac1684dc2a4c1d98b1cc2ad7f. Latest consensus state: current_term: 1 leader_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d356fac1684dc2a4c1d98b1cc2ad7f" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 36133 } } }
I20260807 15:06:27.542307 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f [sys.catalog]: This master's current role is: LEADER
W20260807 15:06:27.542811 24707 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:27.542876 24707 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:27.542969 24706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:27.543205 24706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:27.543769 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:27.542317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36197 (local address 127.1.127.254:36133)
0807 15:06:27.542383 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:27.542385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:27.542566 (+ 181us) server_negotiation.cc:407] Connection header received
0807 15:06:27.542708 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:27.542710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:27.542717 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:27.542786 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:27.543528 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:27.543530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:27.543530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:27.543534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:27.543550 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:27.543555 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:27.543609 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:27.543670 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 15:06:27.543685 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:06:27.545037 24706 catalog_manager.cc:1347] Generated new cluster ID: d3a195bbf30d438fa86088089ea0e596
I20260807 15:06:27.545092 24706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:27.557327 24706 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:27.557556 24706 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:27.567715 24706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7d356fac1684dc2a4c1d98b1cc2ad7f: Generated new TSK 0
I20260807 15:06:27.568011 24706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:27.573562 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24612
2026-08-07T15:06:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1402 ms)
[----------] 18 tests from AuthTokenIssuingTest (25968 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:27 dist-test-slave-8tdl krb5kdc[24713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:27 dist-test-slave-8tdl krb5kdc[24713](info): set up 2 sockets
Aug 07 15:06:27 dist-test-slave-8tdl krb5kdc[24713](info): commencing operation
krb5kdc: starting...
W20260807 15:06:28.695452 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:28 dist-test-slave-8tdl krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:28.840215 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:44073
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:28.929459 24726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:28.929803 24726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:28.929853 24726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:28.929927 24726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:28.931576 24726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:28.931671 24726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:28.931699 24726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:28.931725 24726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:28.931758 24726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:28.933761 24726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:44073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:28.934144 24726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:28.934427 24726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:28.940488 24734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:28.940419 24731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:28.940442 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:28.940622 24726 server_base.cc:1034] running on GCE node
I20260807 15:06:28.940972 24726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:28.941340 24726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:28.942505 24726 hybrid_clock.cc:648] HybridClock initialized: now 1786115188942485 us; error 47 us; skew 500 ppm
Aug 07 15:06:28 dist-test-slave-8tdl krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:28.945580 24726 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:28.946399 24726 webserver.cc:460] Webserver started at http://127.1.127.254:35325/ using document root <none> and password file <none>
I20260807 15:06:28.946666 24726 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:28.946811 24726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:28.946992 24726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:28.948247 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "d0fdfb55435047a39d3fa82a0e5438fd"
format_stamp: "Formatted at 2026-08-07 15:06:28 on dist-test-slave-8tdl"
server_key: "dca376e53d6b250e4a8becaded749011"
server_key_iv: "772a8b4d76f56c8b79b0338ad94fbb6c"
server_key_version: "encryptionkey@0"
I20260807 15:06:28.948669 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "d0fdfb55435047a39d3fa82a0e5438fd"
format_stamp: "Formatted at 2026-08-07 15:06:28 on dist-test-slave-8tdl"
server_key: "dca376e53d6b250e4a8becaded749011"
server_key_iv: "772a8b4d76f56c8b79b0338ad94fbb6c"
server_key_version: "encryptionkey@0"
I20260807 15:06:28.950307 24726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:28.951354 24741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:28.951752 24726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:06:28.951977 24726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "d0fdfb55435047a39d3fa82a0e5438fd"
format_stamp: "Formatted at 2026-08-07 15:06:28 on dist-test-slave-8tdl"
server_key: "dca376e53d6b250e4a8becaded749011"
server_key_iv: "772a8b4d76f56c8b79b0338ad94fbb6c"
server_key_version: "encryptionkey@0"
I20260807 15:06:28.952108 24726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:28.971118 24726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:28.973657 24726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:28.973843 24726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:28.978842 24726 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:44073
I20260807 15:06:28.978910 24793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:44073 every 8 connection(s)
I20260807 15:06:28.979321 24726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:28.980647 24794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:28.982314 24794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:28.987236 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24726
I20260807 15:06:28.987427 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:28.987618 1535 external_mini_cluster.cc:1428] Setting key f6895ccf17410f2460a1c687c75eba3b
Aug 07 15:06:28 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:28.994292 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:28.983190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:44073 (local address 127.0.0.1:43018)
0807 15:06:28.983412 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:28.983460 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:28.984071 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:28.984381 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:28.984383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:28.984715 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:28.984897 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:28.984901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:28.986004 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:28.986005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:28.986746 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:28.986748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:28.987361 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:28.987362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:28.987428 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:28.988081 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:28.988091 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:28.989886 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:28.992413 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:28.992421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:28.992423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:28.992735 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:28.993054 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:28.993055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:28.993055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:28.993145 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:28.993702 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:28.993712 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:28.993807 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 15:06:28.993927 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 15:06:28.994098 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260807 15:06:28.994445 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:28.983438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.1.127.254:44073)
0807 15:06:28.983642 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:28.983647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:28.983683 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:06:28.984256 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:28.984258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:28.984275 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:28.984376 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:28.985012 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:28.985894 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:28.986891 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:28.987228 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:28.987371 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:28.989998 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:28.990008 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:28.990009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:28.990018 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:28.992298 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:28.992843 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:28.992847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:28.992848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:28.992973 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:28.993217 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:28.993218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:28.993218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:28.993595 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:28.993699 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:28.994004 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 15:06:28.994256 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
I20260807 15:06:28.995329 24794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:28.995816 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:28.995662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43786)
0807 15:06:28.995725 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:28.995784 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 15:06:28.996011 24744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
I20260807 15:06:28.996608 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:28.996468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43800)
0807 15:06:28.996544 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:28.996579 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 15:06:28.996829 24794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:44681: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.127.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
I20260807 15:06:29.023427 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:44681
--webserver_interface=127.1.127.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:06:29.046118 24794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } attempt: 1
I20260807 15:06:29.046772 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.046586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43814)
0807 15:06:29.046661 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.046731 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:06:29.047286 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.047173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43818)
0807 15:06:29.047225 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.047261 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:06:29.047396 24794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:44681: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
W20260807 15:06:29.111732 24802 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:29.111990 24802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:29.112011 24802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:29.112035 24802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:29.113300 24802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:29.113354 24802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:29.113369 24802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:29.113384 24802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:29.113395 24802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:29.115034 24802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:44681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.127.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:29.115324 24802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:29.115593 24802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:29.115526 24794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } attempt: 2
I20260807 15:06:29.116061 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.115901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43820)
0807 15:06:29.115977 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.116024 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:06:29.116482 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.116365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43828)
0807 15:06:29.116419 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.116455 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:06:29.116596 24794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:44681: Network error: Client connection negotiation failed: client connection to 127.1.127.253:44681: connect: Connection refused (error 111)
W20260807 15:06:29.121284 24807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:29.121335 24810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:29.121284 24808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:29.121657 24802 server_base.cc:1034] running on GCE node
I20260807 15:06:29.121989 24802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:29.122359 24802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:29.123567 24802 hybrid_clock.cc:648] HybridClock initialized: now 1786115189123524 us; error 63 us; skew 500 ppm
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:29.126540 24802 init.cc:377] Logged in from keytab as kudu/127.1.127.253@KRBTEST.COM (short username kudu)
I20260807 15:06:29.127089 24802 webserver.cc:460] Webserver started at http://127.1.127.253:46743/ using document root <none> and password file <none>
I20260807 15:06:29.127336 24802 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:29.127384 24802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:29.127521 24802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:29.128767 24802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data/instance:
uuid: "be102762e99c47c49aef3c53235aed31"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "f663d85ac746ec5309cf6333d15b84dc"
server_key_iv: "54e5a56f7524298dfb48c1316bbc3c60"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.129159 24802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal/instance:
uuid: "be102762e99c47c49aef3c53235aed31"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "f663d85ac746ec5309cf6333d15b84dc"
server_key_iv: "54e5a56f7524298dfb48c1316bbc3c60"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.130784 24802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:29.131771 24818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:29.132041 24802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:29.132119 24802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "be102762e99c47c49aef3c53235aed31"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "f663d85ac746ec5309cf6333d15b84dc"
server_key_iv: "54e5a56f7524298dfb48c1316bbc3c60"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.132179 24802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:29.142225 24802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:29.144724 24802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:29.144857 24802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:29.149289 24802 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.253:44681
I20260807 15:06:29.149322 24871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.253:44681 every 8 connection(s)
I20260807 15:06:29.149712 24802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
I20260807 15:06:29.150043 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24802
I20260807 15:06:29.150156 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-1/wal/instance
I20260807 15:06:29.150413 1535 external_mini_cluster.cc:1428] Setting key dc49f270ed6cc67923e54919fb71aef6
I20260807 15:06:29.151113 24872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:29.153332 24872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.127.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.252@KRBTEST.COM" created.
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:29.167867 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.154162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.1.127.254:44073)
0807 15:06:29.154235 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.154237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.154469 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:06:29.155136 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.155138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.155156 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.155245 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.156067 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.156953 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.158126 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.158479 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.158568 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.164037 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.164039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.164040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.164051 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.165870 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.166527 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.166528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.166528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.166595 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.167041 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.167042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.167043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.167174 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.167235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.167641 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.167725 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:06:29.168236 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.154113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:44073 (local address 127.0.0.1:43022)
0807 15:06:29.154329 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.154373 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.155020 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.155370 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.155373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.155745 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.155960 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.155965 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.157118 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.157119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.158020 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.158023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.158627 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.158628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.158830 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.159468 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.159512 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.163904 (+ 4392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.166032 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.166045 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.166048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.166424 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.166706 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.166707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.166708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.166899 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.167251 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.167259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.167465 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.167599 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.167816 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":130,"threads_started":1}
I20260807 15:06:29.168964 24872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:29.179206 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.169319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43834)
0807 15:06:29.169378 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.169390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.169437 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.169716 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.169717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.169876 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.169931 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.169932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.170850 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.170852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.171416 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.171418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.171932 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.171933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.172008 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.172458 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.172464 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.175367 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.177671 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.177672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.177673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.178006 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.178266 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.178267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.178267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.178339 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.178832 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.178897 (+ 65us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.178941 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.179007 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.179103 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:06:29.179412 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.169369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.1.127.253:44681)
0807 15:06:29.169576 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.169580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.169597 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:06:29.169615 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.169617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.169628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.169716 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.170007 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.170763 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.171524 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.171854 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.171936 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.175499 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.175510 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.175510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.175517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.177562 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.178112 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.178115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.178116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.178177 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.178432 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.178438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.178439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.178754 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.178830 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.179098 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.179254 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
I20260807 15:06:29.180296 24872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:29.180783 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.180647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57266)
0807 15:06:29.180701 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.180757 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:06:29.181020 24823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
I20260807 15:06:29.181690 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.181580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57274)
0807 15:06:29.181632 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.181666 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:06:29.181833 24872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:41581: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
I20260807 15:06:29.184275 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:41581
--webserver_interface=127.1.127.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:36899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:06:29.207773 24794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } attempt: 3
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:29.216068 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.208298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.1.127.253:44681)
0807 15:06:29.208374 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.208377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.208388 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:29.208505 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.208506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.208518 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.208604 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.208978 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.209493 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.210232 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.210471 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.210564 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.212897 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.212899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.212900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.212980 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.214599 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.215105 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.215106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.215107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.215175 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.215469 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.215471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.215471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.215609 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.215704 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.215867 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.215967 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:06:29.216068 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.208221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43836)
0807 15:06:29.208283 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.208302 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.208376 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.208635 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.208636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.208814 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.208890 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.208892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.209591 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.209592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.210129 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.210130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.210564 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.210565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.210667 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.211243 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.211248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.212774 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.214781 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.214783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.214784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.215013 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.215251 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.215252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.215252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.215319 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.215685 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.215686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.215742 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.215829 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.215956 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:06:29.216521 24794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:29.217013 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.216847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57290)
0807 15:06:29.216940 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.216981 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:06:29.217483 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.217303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57294)
0807 15:06:29.217431 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.217459 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260807 15:06:29.217689 24794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:41581: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
I20260807 15:06:29.231196 24872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } attempt: 1
I20260807 15:06:29.231789 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.231646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57310)
0807 15:06:29.231709 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.231755 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:06:29.232350 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.232230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57326)
0807 15:06:29.232292 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.232326 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:06:29.232445 24872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:41581: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
I20260807 15:06:29.248932 24794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } attempt: 1
I20260807 15:06:29.249570 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.249414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57342)
0807 15:06:29.249486 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.249535 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 15:06:29.249981 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.249865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57350)
0807 15:06:29.249920 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.249952 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:06:29.250094 24794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:41581: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
W20260807 15:06:29.275025 24879 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:29.275323 24879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:29.275357 24879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:29.275404 24879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:29.276679 24879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:29.276772 24879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:29.276794 24879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:29.276814 24879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:29.276854 24879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:29.278548 24879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:36899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:44073,127.1.127.253:44681,127.1.127.252:41581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:41581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.127.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:29.278940 24879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:29.279206 24879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:29.284683 24884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:29.284731 24885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:29.284693 24887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:29.285040 24879 server_base.cc:1034] running on GCE node
I20260807 15:06:29.285334 24879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:29.285662 24879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:29.286804 24879 hybrid_clock.cc:648] HybridClock initialized: now 1786115189286757 us; error 61 us; skew 500 ppm
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:29.289710 24879 init.cc:377] Logged in from keytab as kudu/127.1.127.252@KRBTEST.COM (short username kudu)
I20260807 15:06:29.290303 24879 webserver.cc:460] Webserver started at http://127.1.127.252:33533/ using document root <none> and password file <none>
I20260807 15:06:29.290549 24879 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:29.290621 24879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:29.290799 24879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:29.291931 24879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data/instance:
uuid: "45753a40a6514878bf7b6de6dfbcc539"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "4a09451f2936e2b5fe261ab2f10904e7"
server_key_iv: "1800aa6398d1b404f4991d8788e516b6"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.292330 24879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal/instance:
uuid: "45753a40a6514878bf7b6de6dfbcc539"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "4a09451f2936e2b5fe261ab2f10904e7"
server_key_iv: "1800aa6398d1b404f4991d8788e516b6"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.293851 24879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:29.294937 24894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:29.295163 24879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:29.295288 24879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "45753a40a6514878bf7b6de6dfbcc539"
format_stamp: "Formatted at 2026-08-07 15:06:29 on dist-test-slave-8tdl"
server_key: "4a09451f2936e2b5fe261ab2f10904e7"
server_key_iv: "1800aa6398d1b404f4991d8788e516b6"
server_key_version: "encryptionkey@0"
I20260807 15:06:29.295375 24879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:29.300606 24872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } attempt: 2
I20260807 15:06:29.301165 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.301016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57362)
0807 15:06:29.301083 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.301130 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:06:29.301652 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.301514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57368)
0807 15:06:29.301590 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.301627 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 15:06:29.301795 24872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:41581: Network error: Client connection negotiation failed: client connection to 127.1.127.252:41581: connect: Connection refused (error 111)
I20260807 15:06:29.316937 24879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:29.319846 24879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:29.320137 24879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:29.324720 24879 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.252:41581
I20260807 15:06:29.324818 24947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.252:41581 every 8 connection(s)
I20260807 15:06:29.325255 24879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
I20260807 15:06:29.325243 24794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } attempt: 2
I20260807 15:06:29.326196 24948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:29.328835 24948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:29.330935 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 24879
I20260807 15:06:29.331043 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786114902562993-1535-0/minicluster-data/master-2/wal/instance
I20260807 15:06:29.331246 1535 external_mini_cluster.cc:1428] Setting key 60236f35031cc89fd40c3098db232ecd
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:06:29.339541 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.325618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57374)
0807 15:06:29.325686 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.325698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.325749 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.327361 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.327362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.327580 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.327630 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.327633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.329343 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.329345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.331078 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.331082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.332069 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.332070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.332135 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.333012 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.333016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.335142 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.337478 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.337479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.337480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.337855 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.338197 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.338198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.338198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.338276 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.338846 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.338848 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.338907 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.339017 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.339434 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:06:29.339668 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.329617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.1.127.254:44073)
0807 15:06:29.329670 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.329673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.329841 (+ 168us) server_negotiation.cc:407] Connection header received
0807 15:06:29.329905 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.329906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.329915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.329974 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.330767 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.331467 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.332657 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.332886 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.332947 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.335671 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.335673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.335674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.335682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.337860 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.338463 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.338464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.338464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.338533 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.338812 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.338813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.338814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.339002 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.339087 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.339375 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.339515 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":30}
I20260807 15:06:29.339969 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.329505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:44073 (local address 127.0.0.1:43030)
0807 15:06:29.329710 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.329764 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.329847 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.330002 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.330005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.330439 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.330596 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.330599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.331586 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.331587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.332576 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.332579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.333085 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.333087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.333167 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.333691 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.333701 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.335548 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.337982 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.337987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.337990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.338324 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.338609 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.338610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.338611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.338745 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.339087 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.339096 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.339173 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.339275 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.339502 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":110,"threads_started":1}
I20260807 15:06:29.340227 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.326283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57374 (local address 127.1.127.252:41581)
0807 15:06:29.326504 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.326512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.326545 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:06:29.327114 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.327123 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.327192 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.327359 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.327750 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.329163 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.331195 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.331987 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.332114 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.335301 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.335311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.335311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.335320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.337356 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.337960 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.337964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.337965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.338109 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.338367 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.338368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.338368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.338752 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.339686 (+ 934us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.339750 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.340111 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
I20260807 15:06:29.340561 24948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:29.342139 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: Bootstrap starting.
I20260807 15:06:29.342567 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.331777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.1.127.252:41581)
0807 15:06:29.331936 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.331938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.332092 (+ 154us) server_negotiation.cc:407] Connection header received
0807 15:06:29.332193 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.332193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.332204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.332246 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.333085 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.333908 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.335031 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.335290 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.335365 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.338859 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.338860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.338861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.338868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.341053 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.341575 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.341576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.341576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.341636 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.341866 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.341868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.341868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.341992 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.342159 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.342300 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.342395 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260807 15:06:29.342998 24794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:29.343338 24794 log.cc:824] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:29.344298 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: No bootstrap required, opened a new log
I20260807 15:06:29.345860 24794 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.346040 24794 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0fdfb55435047a39d3fa82a0e5438fd, State: Initialized, Role: FOLLOWER
I20260807 15:06:29.346138 24794 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.346443 24955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.346530 24794 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:29.346542 24955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
W20260807 15:06:29.348771 24966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:29.348831 24966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:29.352387 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.343187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.1.127.254:44073)
0807 15:06:29.343225 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.343227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.343267 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:06:29.343329 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.343330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.343340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.343389 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.344076 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.344610 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.345348 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.345549 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.345623 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.348617 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.348619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.348620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.348629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.350894 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.351369 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.351371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.351372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.351513 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.351768 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.351769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.351769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.351919 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.352075 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.352192 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.352290 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:06:29.350584 24726 master_runner.cc:186] Error getting master registration for 127.1.127.253:44681: OK, instance_id { permanent_uuid: "be102762e99c47c49aef3c53235aed31" instance_seqno: 1786115189148063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:06:29.353581 24726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:06:29.355949 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.340854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:44681 (local address 127.0.0.1:43844)
0807 15:06:29.340942 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.341012 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.341065 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.341210 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.341211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.341383 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.341435 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.341438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.342315 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.342317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.343109 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.343112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.343675 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.343676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.343736 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.346833 (+ 3097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.346840 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.348768 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.354608 (+ 5840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.354610 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.354611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.354929 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.355190 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.355191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.355191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.355266 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.355582 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.355583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.355627 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.355707 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.355827 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 15:06:29.356038 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.340931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.1.127.253:44681)
0807 15:06:29.340988 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.340990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.341070 (+ 80us) server_negotiation.cc:407] Connection header received
0807 15:06:29.341128 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.341129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.341142 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.341196 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.341523 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.342212 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.343227 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.343539 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.343591 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.349579 (+ 5988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.349582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.349582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.349591 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.354505 (+ 4914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.355029 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.355031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.355031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.355106 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.355352 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.355353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.355353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.355510 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.355583 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.355757 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.355858 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:06:29.356668 24948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:29.364006 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.356961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57382)
0807 15:06:29.357061 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.357073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.357126 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.357276 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.357277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.357429 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.357460 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.357462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.358106 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.358107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.358628 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.358630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.359076 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.359076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.359123 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.359610 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.359615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.360844 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.362797 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.362799 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.362799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.363041 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.363271 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.363271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.363272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.363340 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.363700 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.363701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.363737 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.363796 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.363897 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 15:06:29.364372 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.357024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.1.127.252:41581)
0807 15:06:29.357064 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.357067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.357110 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:06:29.357213 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.357214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.357225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.357274 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.357538 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.358023 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.358762 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.358988 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.359075 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.360967 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.360968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.360968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.360975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.362682 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.363127 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.363129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.363129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.363195 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.363416 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.363416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.363417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.363617 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.363678 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.364212 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.364288 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:06:29.365693 24948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: Bootstrap starting.
I20260807 15:06:29.366364 24948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:29.366731 24948 log.cc:824] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:29.367213 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.354278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.1.127.253:44681)
0807 15:06:29.354523 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.354525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.354534 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:29.354558 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.354559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.354570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.354648 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.355610 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.357493 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.358540 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.358923 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.359340 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.362019 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.362020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.362020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.362029 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.363798 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.364210 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.364212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.364212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.364274 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.364499 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.364500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.364501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.364634 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.364800 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.366965 (+ 2165us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.367082 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":154,"threads_started":1}
I20260807 15:06:29.367959 24948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: No bootstrap required, opened a new log
I20260807 15:06:29.368755 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:29.369935 24768 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d0fdfb55435047a39d3fa82a0e5438fd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046
I20260807 15:06:29.370110 24768 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:06:29.370158 24768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:29.370182 24948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.370307 24948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45753a40a6514878bf7b6de6dfbcc539, State: Initialized, Role: FOLLOWER
I20260807 15:06:29.370368 24948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.370651 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.370754 24948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:29.370765 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:29.371230 24768 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.371492 24768 leader_election.cc:288] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [CANDIDATE]: Term 1 election: Requested vote from peers be102762e99c47c49aef3c53235aed31 (127.1.127.253:44681), 45753a40a6514878bf7b6de6dfbcc539 (127.1.127.252:41581)
I20260807 15:06:29.372237 24922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "45753a40a6514878bf7b6de6dfbcc539"
I20260807 15:06:29.372351 24922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [term 0 FOLLOWER]: Advancing to term 1
W20260807 15:06:29.372673 24982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:29.374563 24982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:29.373301 24922 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0fdfb55435047a39d3fa82a0e5438fd in term 1.
I20260807 15:06:29.374939 24745 leader_election.cc:302] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45753a40a6514878bf7b6de6dfbcc539, d0fdfb55435047a39d3fa82a0e5438fd; no voters:
I20260807 15:06:29.375169 24955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:29.375339 24955 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [term 1 LEADER]: Becoming Leader. State: Replica: d0fdfb55435047a39d3fa82a0e5438fd, State: Running, Role: LEADER
I20260807 15:06:29.375444 24955 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.375872 24955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0fdfb55435047a39d3fa82a0e5438fd. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.375955 24955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:29.377066 24879 master_runner.cc:186] Error getting master registration for 127.1.127.253:44681: OK, instance_id { permanent_uuid: "be102762e99c47c49aef3c53235aed31" instance_seqno: 1786115189148063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:06:29.379498 24879 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:06:29.384723 24996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:29.385039 24996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:29.387259 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.374565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.1.127.252:41581)
0807 15:06:29.374826 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.374829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.376216 (+ 1387us) server_negotiation.cc:407] Connection header received
0807 15:06:29.376384 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.376385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.376397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.376428 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.379797 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.380552 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.381686 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.382002 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.382073 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.383890 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.383892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.383892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.383901 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.385753 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.386249 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.386250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.386251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.386315 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.386538 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.386540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.386540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.386718 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.386835 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.387083 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.387164 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260807 15:06:29.387460 24922 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [term 1 FOLLOWER]: Refusing update from remote peer d0fdfb55435047a39d3fa82a0e5438fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:29.388775 24993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:29.390321 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0fdfb55435047a39d3fa82a0e5438fd. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.390406 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:29.392789 24993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.392890 24993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:29.393217 24872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } attempt: 3
I20260807 15:06:29.393301 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.393380 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:29.393721 24997 mvcc.cc:205] Tried to move back new op lower bound from 7315927815727263744 to 7315927815682535424. Current Snapshot: MvccSnapshot[applied={T|T < 7315927815727263744 or (T in {7315927815727263744})}]
I20260807 15:06:29.394106 24998 mvcc.cc:205] Tried to move back new op lower bound from 7315927815727263744 to 7315927815682535424. Current Snapshot: MvccSnapshot[applied={T|T < 7315927815727263744}]
I20260807 15:06:29.394925 24996 catalog_manager.cc:1347] Generated new cluster ID: d57573a380f84a0cb4b07b4fc6906229
I20260807 15:06:29.394980 24996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:29.396185 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.375109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.1.127.254:44073)
0807 15:06:29.375180 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.375182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.386202 (+ 11020us) server_negotiation.cc:407] Connection header received
0807 15:06:29.386310 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.386312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.386325 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.386359 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.387083 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.387833 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.388861 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.389144 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.389188 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.390780 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.390781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.390782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.390791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.394771 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.395309 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.395311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.395311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.395374 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.395573 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.395573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.395574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.395706 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.395789 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.395994 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.396094 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 15:06:29 dist-test-slave-8tdl krb5kdc[24713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:06:29.398283 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.374715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.1.127.253:44681)
0807 15:06:29.374944 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.374946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.387599 (+ 12653us) server_negotiation.cc:407] Connection header received
0807 15:06:29.387696 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.387698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.387709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.387738 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.388416 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.389147 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.390134 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.390527 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.390572 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.392247 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.392248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.392249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.392256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.394658 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.397307 (+ 2649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.397308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.397308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.397379 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.397600 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.397601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.397601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.397735 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.397818 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.398091 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.398186 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260807 15:06:29.401839 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.393474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:41581 (local address 127.0.0.1:57408)
0807 15:06:29.393579 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:29.393590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:29.393634 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:29.393785 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:29.393785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:29.393999 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:29.394070 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.394073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.395114 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.395115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.395935 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:29.395937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.396518 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.396519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:29.396589 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.396982 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:29.396987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:29.398522 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:29.400731 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.400732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.400733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.400954 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.401175 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:29.401176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:29.401176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:29.401240 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:29.401517 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:29.401518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:29.401571 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:06:29.401636 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:06:29.401741 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 15:06:29.402196 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:29.393536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.1.127.252:41581)
0807 15:06:29.393577 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.393594 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.393685 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:06:29.393706 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.393707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.393716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.393766 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.394190 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.394998 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.396101 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:29.396402 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:29.396453 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:29.398741 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:29.398743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:29.398743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:29.398750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:29.400626 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.401032 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.401033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.401034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.401102 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:29.401303 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:29.401304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:29.401304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:29.401437 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:29.401485 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:29.402017 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 15:06:29.402110 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:06:29.403816 24872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: Bootstrap starting.
I20260807 15:06:29.404572 24872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:29.404860 24872 log.cc:824] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:29.411548 24872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: No bootstrap required, opened a new log
I20260807 15:06:29.413785 24872 raft_consensus.cc:348] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.413947 24872 raft_consensus.cc:729] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be102762e99c47c49aef3c53235aed31, State: Initialized, Role: FOLLOWER
I20260807 15:06:29.414048 24872 consensus_queue.cc:260] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } }
I20260807 15:06:29.414346 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.414418 24872 sys_catalog.cc:564] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:29.414512 24845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:29.416118 24845 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [term 1 FOLLOWER]: Refusing update from remote peer d0fdfb55435047a39d3fa82a0e5438fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:29.416167 24846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "be102762e99c47c49aef3c53235aed31"
I20260807 15:06:29.416005 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:29.417140 24846 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0fdfb55435047a39d3fa82a0e5438fd in term 1.
I20260807 15:06:29.417454 24955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:29.419128 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0fdfb55435047a39d3fa82a0e5438fd. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.419292 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:29.419373 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.419478 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:06:29.420473 25014 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:29.420549 25014 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:29.423436 25015 mvcc.cc:205] Tried to move back new op lower bound from 7315927815727263744 to 7315927815682535424. Current Snapshot: MvccSnapshot[applied={T|T < 7315927815727263744}]
I20260807 15:06:29.424873 24993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0fdfb55435047a39d3fa82a0e5438fd" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 44073 } } peers { permanent_uuid: "be102762e99c47c49aef3c53235aed31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 44681 } } peers { permanent_uuid: "45753a40a6514878bf7b6de6dfbcc539" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 41581 } } }
I20260807 15:06:29.424973 24993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:29.430786 24996 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:29.431738 24996 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:29.441361 24996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0fdfb55435047a39d3fa82a0e5438fd: Generated new TSK 0
I20260807 15:06:29.441666 24996 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:06:29.477948 24952 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.475885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.1.127.252:41581)
0807 15:06:29.475935 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.475937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.477116 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 15:06:29.477241 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.477242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.477256 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.477290 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.477864 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57414: BlockingRecv error: recv got EOF from 127.0.0.1:57414 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:06:29.479208 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.476364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.1.127.253:44681)
0807 15:06:29.476407 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.476410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.478380 (+ 1970us) server_negotiation.cc:407] Connection header received
0807 15:06:29.478559 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.478560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.478573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.478605 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.479139 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43872: BlockingRecv error: recv got EOF from 127.0.0.1:43872 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:06:29.480026 24798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.476767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.1.127.254:44073)
0807 15:06:29.476810 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.476812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.478733 (+ 1921us) server_negotiation.cc:407] Connection header received
0807 15:06:29.479580 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.479582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.479594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.479625 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.479949 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43060: BlockingRecv error: recv got EOF from 127.0.0.1:43060 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:06:29.483258 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.482296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.1.127.253:44681)
0807 15:06:29.482336 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.482338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.482570 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:06:29.482742 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.482743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.482767 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.482804 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.483192 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43888: BlockingRecv error: recv got EOF from 127.0.0.1:43888 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:06:29.494896 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.491211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.1.127.253:44681)
0807 15:06:29.491260 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.491262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.494119 (+ 2857us) server_negotiation.cc:407] Connection header received
0807 15:06:29.494247 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.494249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.494264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.494296 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.494831 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43904: BlockingRecv error: recv got EOF from 127.0.0.1:43904 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:06:29.499965 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.498714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.1.127.253:44681)
0807 15:06:29.498766 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.498768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.499077 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:06:29.499184 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.499185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.499199 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.499229 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.499660 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43906: BlockingRecv error: recv got EOF from 127.0.0.1:43906 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:06:29.506433 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.505343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.1.127.253:44681)
0807 15:06:29.505389 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.505391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.505769 (+ 378us) server_negotiation.cc:407] Connection header received
0807 15:06:29.505931 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.505932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.505948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.505982 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.506374 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43918: BlockingRecv error: recv got EOF from 127.0.0.1:43918 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:06:29.517019 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.516011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.1.127.253:44681)
0807 15:06:29.516057 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.516059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.516330 (+ 271us) server_negotiation.cc:407] Connection header received
0807 15:06:29.516441 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.516443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.516457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.516489 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.516952 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43934: BlockingRecv error: recv got EOF from 127.0.0.1:43934 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:06:29.535632 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.534572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.1.127.253:44681)
0807 15:06:29.534631 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.534633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.534992 (+ 359us) server_negotiation.cc:407] Connection header received
0807 15:06:29.535099 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.535100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.535111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.535142 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.535568 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43940: BlockingRecv error: recv got EOF from 127.0.0.1:43940 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 15:06:29.570011 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.569201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.1.127.253:44681)
0807 15:06:29.569250 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.569252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.569400 (+ 148us) server_negotiation.cc:407] Connection header received
0807 15:06:29.569477 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.569479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.569492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.569569 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.569930 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43944: BlockingRecv error: recv got EOF from 127.0.0.1:43944 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:06:29.636222 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.635474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.1.127.253:44681)
0807 15:06:29.635545 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.635547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.635577 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:06:29.635661 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.635662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.635673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.635724 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.636150 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43960: BlockingRecv error: recv got EOF from 127.0.0.1:43960 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 15:06:29.766355 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:29.765496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.1.127.253:44681)
0807 15:06:29.765579 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:29.765581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:29.765599 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:06:29.765655 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:29.765656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:29.765666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:29.765751 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:29.766227 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43962: BlockingRecv error: recv got EOF from 127.0.0.1:43962 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 15:06:30.024672 24968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:30.023921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.1.127.253:44681)
0807 15:06:30.024032 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.024034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.024047 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:06:30.024081 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.024082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.024099 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.024169 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:30.024599 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43972: BlockingRecv error: recv got EOF from 127.0.0.1:43972 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 15:06:30.375113 24982 catalog_manager.cc:1259] Loaded cluster ID: d57573a380f84a0cb4b07b4fc6906229
I20260807 15:06:30.375207 24982 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: loading cluster ID for follower catalog manager: success
I20260807 15:06:30.376091 24982 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: acquiring CA information for follower catalog manager: success
I20260807 15:06:30.376435 24982 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45753a40a6514878bf7b6de6dfbcc539: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:06:30.421062 25014 catalog_manager.cc:1259] Loaded cluster ID: d57573a380f84a0cb4b07b4fc6906229
I20260807 15:06:30.421198 25014 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: loading cluster ID for follower catalog manager: success
I20260807 15:06:30.422137 25014 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: acquiring CA information for follower catalog manager: success
I20260807 15:06:30.422497 25014 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be102762e99c47c49aef3c53235aed31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:06:30.541190 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:30.538216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.1.127.253:44681)
0807 15:06:30.538398 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.538400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.538413 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:06:30.538491 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.538492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.538560 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.538656 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:30.538939 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.539474 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.540279 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.540507 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.540585 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:30.540795 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:30.540914 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:30.541007 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:30.541053 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 15:06:30.541077 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":61,"threads_started":1}
I20260807 15:06:30.545791 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:30.543284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.1.127.252:41581)
0807 15:06:30.543421 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.543423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.543432 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:30.543471 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.543472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.543524 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.543595 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:30.543663 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.544157 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.544996 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.545201 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.545259 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:30.545423 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:30.545536 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:30.545595 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:30.545698 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:06:30.545720 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260807 15:06:30.550796 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:30.547694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43990 (local address 127.1.127.253:44681)
0807 15:06:30.547739 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.547742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.548098 (+ 356us) server_negotiation.cc:407] Connection header received
0807 15:06:30.548138 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.548140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.548153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.548206 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:30.548383 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.548865 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.549951 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.550258 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.550327 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:30.550477 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:30.550535 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:30.550586 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:30.550655 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 15:06:30.550713 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:06:30.551398 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:30.547745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.1.127.252:41581)
0807 15:06:30.547785 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.547786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.548098 (+ 312us) server_negotiation.cc:407] Connection header received
0807 15:06:30.548209 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.548210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.548219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.548270 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:30.548418 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.549145 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.550378 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.550699 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.550915 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:30.551110 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:30.551152 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:30.551208 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:30.551288 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:06:30.551326 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:30.551406 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:30.547710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.1.127.254:44073)
0807 15:06:30.547832 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:30.547834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:30.547842 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:30.548027 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:30.548029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:30.548090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:30.548184 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:30.548303 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.549117 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.550121 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:30.550431 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:30.550521 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:30.550780 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:30.550905 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:30.551226 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:30.551244 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 15:06:30.551279 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260807 15:06:30.553385 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24726
I20260807 15:06:30.560355 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24802
I20260807 15:06:30.568012 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 24879
2026-08-07T15:06:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2997 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:30 dist-test-slave-8tdl krb5kdc[25103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:30 dist-test-slave-8tdl krb5kdc[25103](info): set up 2 sockets
Aug 07 15:06:30 dist-test-slave-8tdl krb5kdc[25103](info): commencing operation
krb5kdc: starting...
W20260807 15:06:31.728875 1535 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/test-user.keytab.
Aug 07 15:06:31 dist-test-slave-8tdl krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.127.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
Entry for principal kudu/127.1.127.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab.
I20260807 15:06:31.871889 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37827
--webserver_interface=127.1.127.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:31.960327 25118 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:31.960587 25118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:31.960608 25118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:31.960631 25118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:31.961887 25118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:31.961934 25118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:31.961949 25118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:31.961962 25118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:31.961975 25118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:31.963617 25118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.254.keytab
--principal=kudu/127.1.127.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.254:37827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.127.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:31.963866 25118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:31.964064 25118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:31.969439 25127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:31.969703 25124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:31.969703 25125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:31.969728 25118 server_base.cc:1034] running on GCE node
I20260807 15:06:31.970155 25118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:31.970510 25118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:31.971714 25118 hybrid_clock.cc:648] HybridClock initialized: now 1786115191971690 us; error 47 us; skew 500 ppm
Aug 07 15:06:31 dist-test-slave-8tdl krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:31.980252 25118 init.cc:377] Logged in from keytab as kudu/127.1.127.254@KRBTEST.COM (short username kudu)
I20260807 15:06:31.980795 25118 webserver.cc:460] Webserver started at http://127.1.127.254:45999/ using document root <none> and password file <none>
I20260807 15:06:31.981056 25118 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:31.981132 25118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:31.981293 25118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:31.982465 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data/instance:
uuid: "0b81546d18e240d2a4320cc7166208e4"
format_stamp: "Formatted at 2026-08-07 15:06:31 on dist-test-slave-8tdl"
server_key: "70c392d5c5356f7b1816bb36b560ec93"
server_key_iv: "44ae0f2644caf44e142aa32a78449c45"
server_key_version: "encryptionkey@0"
I20260807 15:06:31.982877 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance:
uuid: "0b81546d18e240d2a4320cc7166208e4"
format_stamp: "Formatted at 2026-08-07 15:06:31 on dist-test-slave-8tdl"
server_key: "70c392d5c5356f7b1816bb36b560ec93"
server_key_iv: "44ae0f2644caf44e142aa32a78449c45"
server_key_version: "encryptionkey@0"
I20260807 15:06:31.984444 25118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:31.985412 25134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:31.985786 25118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:31.985905 25118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal
uuid: "0b81546d18e240d2a4320cc7166208e4"
format_stamp: "Formatted at 2026-08-07 15:06:31 on dist-test-slave-8tdl"
server_key: "70c392d5c5356f7b1816bb36b560ec93"
server_key_iv: "44ae0f2644caf44e142aa32a78449c45"
server_key_version: "encryptionkey@0"
I20260807 15:06:31.986066 25118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:32.019157 25118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:32.021919 25118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:32.022135 25118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:32.026926 25118 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.254:37827
I20260807 15:06:32.026976 25186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.254:37827 every 8 connection(s)
I20260807 15:06:32.027371 25118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:32.028434 25187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:32.029364 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 25118
I20260807 15:06:32.029451 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-0/wal/instance
I20260807 15:06:32.029680 1535 external_mini_cluster.cc:1428] Setting key 5ae9b8ffef1f4551323c911c9f4ac6b9
I20260807 15:06:32.030258 25187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.127.253@KRBTEST.COM; defaulting to no policy
I20260807 15:06:32.042038 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.031106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37827 (local address 127.0.0.1:34140)
0807 15:06:32.031302 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.031339 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.031807 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.032110 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.032112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.032411 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.032603 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.032608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.033650 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.033651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.034501 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.034505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.035395 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.035397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.035536 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.036174 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.036183 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.038062 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.040408 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.040418 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.040421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.040627 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.040966 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.040967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.040967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.041060 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.041439 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.041444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.041505 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.041583 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.041755 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260807 15:06:32.042047 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.031139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.1.127.254:37827)
0807 15:06:32.031379 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.031384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.031420 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:06:32.031987 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.031990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.032007 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.032105 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.032714 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.033538 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.034604 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.035238 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.035376 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.038170 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.038179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.038180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.038189 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.040247 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.040731 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.040734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.040734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.040825 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.041129 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.041130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.041130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.041369 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.041442 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.041663 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.041906 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.253@KRBTEST.COM" created.
I20260807 15:06:32.042943 25187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:32.043416 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.043253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:36978)
0807 15:06:32.043313 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.043371 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 15:06:32.043619 25135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
I20260807 15:06:32.044149 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.044031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:36992)
0807 15:06:32.044084 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.044117 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:06:32.044340 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:32837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
Entry for principal kudu/127.1.127.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab.
I20260807 15:06:32.063167 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:32837
--webserver_interface=127.1.127.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:06:32.093657 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } attempt: 1
I20260807 15:06:32.094404 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.094205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37006)
0807 15:06:32.094306 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.094362 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 15:06:32.095081 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.094832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37018)
0807 15:06:32.095017 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.095053 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 15:06:32.095234 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:32837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
W20260807 15:06:32.152451 25194 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:32.152706 25194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:32.152725 25194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:32.152748 25194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:32.153949 25194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:32.153995 25194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:32.154009 25194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:32.154021 25194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:32.154033 25194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:32.155682 25194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.253.keytab
--principal=kudu/127.1.127.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.253:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.127.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:32.155930 25194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:32.156145 25194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:32.161993 25201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:32.162021 25203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:32.161932 25199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:32.162400 25194 server_base.cc:1034] running on GCE node
I20260807 15:06:32.162747 25194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:32.163218 25194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:32.163480 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } attempt: 2
I20260807 15:06:32.163918 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.163778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37024)
0807 15:06:32.163846 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.163884 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:06:32.164438 25194 hybrid_clock.cc:648] HybridClock initialized: now 1786115192164419 us; error 60 us; skew 500 ppm
I20260807 15:06:32.164630 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.164425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37032)
0807 15:06:32.164496 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.164528 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:06:32.164791 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.253:32837: Network error: Client connection negotiation failed: client connection to 127.1.127.253:32837: connect: Connection refused (error 111)
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:32.167425 25194 init.cc:377] Logged in from keytab as kudu/127.1.127.253@KRBTEST.COM (short username kudu)
I20260807 15:06:32.167917 25194 webserver.cc:460] Webserver started at http://127.1.127.253:37887/ using document root <none> and password file <none>
I20260807 15:06:32.168144 25194 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:32.168186 25194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:32.168285 25194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:32.169373 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data/instance:
uuid: "1641b69c445047a4878277c947a53b31"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "5788982948af94de32e36359de566325"
server_key_iv: "861a2239b98e37e7013b4ced22127e04"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.169700 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal/instance:
uuid: "1641b69c445047a4878277c947a53b31"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "5788982948af94de32e36359de566325"
server_key_iv: "861a2239b98e37e7013b4ced22127e04"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.171175 25194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:32.171962 25210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:32.172186 25194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:32.172277 25194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal
uuid: "1641b69c445047a4878277c947a53b31"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "5788982948af94de32e36359de566325"
server_key_iv: "861a2239b98e37e7013b4ced22127e04"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.172339 25194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:32.184821 25194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:32.187412 25194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:32.187554 25194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:32.192111 25194 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.253:32837
I20260807 15:06:32.192147 25265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.253:32837 every 8 connection(s)
I20260807 15:06:32.192550 25194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/data/info.pb
I20260807 15:06:32.193712 25266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:32.195604 25266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:32.200109 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 25194
I20260807 15:06:32.200248 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-1/wal/instance
I20260807 15:06:32.200466 1535 external_mini_cluster.cc:1428] Setting key 7da2b2036285bef418c94973f47c490f
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:32.206947 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.196567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.1.127.254:37827)
0807 15:06:32.196658 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.196660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.196998 (+ 338us) server_negotiation.cc:407] Connection header received
0807 15:06:32.197619 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.197620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.197634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.197689 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.198532 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.199104 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.200153 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.200449 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.200528 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.203150 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.203152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.203153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.203163 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.205088 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.205557 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.205558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.205559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.205631 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.205960 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.205961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.205962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.206130 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.206246 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.206791 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.206873 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:06:32.207262 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.196535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37827 (local address 127.0.0.1:34156)
0807 15:06:32.196791 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.196833 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.197489 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.197779 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.197783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.198157 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.198420 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.198426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.199212 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.199213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.200055 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.200058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.200563 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.200565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.200696 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.201283 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.201292 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.203040 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.205206 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.205214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.205216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.205472 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.205728 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.205729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.205729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.205877 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.206258 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.206266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.206537 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.206734 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.206951 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":123,"threads_started":1}
I20260807 15:06:32.208011 25266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.127.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.127.252@KRBTEST.COM" created.
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:32.218366 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.208328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37044)
0807 15:06:32.208392 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.208407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.208466 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.208745 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.208746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.208922 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.208998 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.209001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.209896 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.209897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.210804 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.210808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.211310 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.211311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.211381 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.211926 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.211931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.214915 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.217050 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.217051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.217051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.217230 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.217525 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.217526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.217527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.217589 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.217906 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.217907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.217957 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.218063 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.218238 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:06:32.218400 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.208370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.1.127.253:32837)
0807 15:06:32.208585 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.208590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.208610 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:06:32.208639 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.208641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.208652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.208716 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.209082 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.209791 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.210892 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.211231 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.211291 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.215030 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.215047 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.215051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.215059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.216930 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.217336 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.217339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.217340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.217403 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.217655 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.217656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.217656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.217825 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.217918 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.218101 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.218302 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":186,"thread_start_us":149,"threads_started":1}
I20260807 15:06:32.219084 25266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:32.219498 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.219362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:45992)
0807 15:06:32.219414 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.219463 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 15:06:32.219635 25211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
I20260807 15:06:32.220095 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.219997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46002)
0807 15:06:32.220047 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.220071 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:06:32.220209 25266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
Entry for principal kudu/127.1.127.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab.
I20260807 15:06:32.235165 1535 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbmaIkX/build/release/bin/kudu
/tmp/dist-test-taskbmaIkX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:46019
--webserver_interface=127.1.127.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.127.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:06:32.255992 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } attempt: 3
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:32.265120 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.256413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37050)
0807 15:06:32.256564 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.256578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.256640 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.256856 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.256858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.257106 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.257187 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.257190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.258170 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.258172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.258984 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.258988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.259538 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.259539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.259612 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.260163 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.260169 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.261667 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.263763 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.263764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.263764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.263963 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.264210 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.264211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.264211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.264287 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.264768 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.264769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.264841 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.264920 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.265033 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 15:06:32.265168 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.256519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.1.127.253:32837)
0807 15:06:32.256584 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.256587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.256648 (+ 61us) server_negotiation.cc:407] Connection header received
0807 15:06:32.256689 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.256690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.256703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.256790 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.257285 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.258053 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.259079 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.259419 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.259504 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.261775 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.261777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.261778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.261787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.263636 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.264042 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.264043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.264044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.264114 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.264364 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.264365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.264365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.264633 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.264724 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.264973 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.265072 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:06:32.265653 25187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:32.266036 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.265911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46008)
0807 15:06:32.265969 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.266006 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:06:32.266414 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.266304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46014)
0807 15:06:32.266355 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.266383 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 15:06:32.266515 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
I20260807 15:06:32.269454 25266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 1
I20260807 15:06:32.269912 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.269777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46030)
0807 15:06:32.269841 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.269877 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:06:32.270365 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.270263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46042)
0807 15:06:32.270313 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.270340 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 15:06:32.270471 25266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
I20260807 15:06:32.297667 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 1
I20260807 15:06:32.298446 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.298243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46052)
0807 15:06:32.298350 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.298404 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 15:06:32.298904 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.298811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46062)
0807 15:06:32.298849 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.298879 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:06:32.299042 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
W20260807 15:06:32.328045 25273 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:06:32.328286 25273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:32.328310 25273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:32.328333 25273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:32.329651 25273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:32.329712 25273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:32.329727 25273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:32.329741 25273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:32.329753 25273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:32.331423 25273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.127.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.127.254:37827,127.1.127.253:32837,127.1.127.252:46019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/krb5kdc/kudu_127.1.127.252.keytab
--principal=kudu/127.1.127.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.127.252:46019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.127.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:32.331707 25273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:32.331990 25273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:32.337733 25281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:32.337719 25279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:32.337719 25278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:32.337828 25273 server_base.cc:1034] running on GCE node
I20260807 15:06:32.338294 25273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:32.338640 25273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:32.338655 25266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 2
I20260807 15:06:32.339100 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.338961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46064)
0807 15:06:32.339028 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.339058 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 15:06:32.339409 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.339318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46070)
0807 15:06:32.339363 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.339387 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:06:32.339516 25266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
I20260807 15:06:32.339855 25273 hybrid_clock.cc:648] HybridClock initialized: now 1786115192339838 us; error 36 us; skew 500 ppm
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:32.342880 25273 init.cc:377] Logged in from keytab as kudu/127.1.127.252@KRBTEST.COM (short username kudu)
I20260807 15:06:32.343380 25273 webserver.cc:460] Webserver started at http://127.1.127.252:39713/ using document root <none> and password file <none>
I20260807 15:06:32.343688 25273 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:32.343735 25273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:32.343907 25273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:32.345108 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data/instance:
uuid: "04bc8df61d57411bbad0a991b8d76b12"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "69120e8980dd84f17bea61f3f72dbd0b"
server_key_iv: "499ea690ba01f18ba6795f31b66570a9"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.345505 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal/instance:
uuid: "04bc8df61d57411bbad0a991b8d76b12"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "69120e8980dd84f17bea61f3f72dbd0b"
server_key_iv: "499ea690ba01f18ba6795f31b66570a9"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.347110 25273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 15:06:32.348256 25288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:32.348542 25273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:32.348662 25273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data,/tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal
uuid: "04bc8df61d57411bbad0a991b8d76b12"
format_stamp: "Formatted at 2026-08-07 15:06:32 on dist-test-slave-8tdl"
server_key: "69120e8980dd84f17bea61f3f72dbd0b"
server_key_iv: "499ea690ba01f18ba6795f31b66570a9"
server_key_version: "encryptionkey@0"
I20260807 15:06:32.348762 25273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:32.371726 25273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:32.374230 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 2
I20260807 15:06:32.374503 25273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:32.374744 25273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:32.374771 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.374569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46078)
0807 15:06:32.374643 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.374729 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:06:32.375371 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.375246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46088)
0807 15:06:32.375306 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.375338 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 15:06:32.375560 25187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.127.252:46019: Network error: Client connection negotiation failed: client connection to 127.1.127.252:46019: connect: Connection refused (error 111)
I20260807 15:06:32.379742 25273 rpc_server.cc:307] RPC server started. Bound to: 127.1.127.252:46019
I20260807 15:06:32.379776 25340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.127.252:46019 every 8 connection(s)
I20260807 15:06:32.380203 25273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/data/info.pb
I20260807 15:06:32.381346 25341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:32.381798 1535 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbmaIkX/build/release/bin/kudu as pid 25273
I20260807 15:06:32.381929 1535 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbmaIkX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786114902562993-1535-0/minicluster-data/master-2/wal/instance
I20260807 15:06:32.382171 1535 external_mini_cluster.cc:1428] Setting key 433824a3aaf7aedb51c04bd9dd079721
I20260807 15:06:32.383658 25341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:06:32.394459 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.384550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.1.127.254:37827)
0807 15:06:32.384609 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.384611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.384816 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:06:32.384928 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.384929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.384942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.384995 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.385669 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.386251 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.387401 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.387600 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.387661 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.390581 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.390583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.390583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.390593 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.392411 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.393053 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.393055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.393055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.393138 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.393452 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.393453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.393454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.393636 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.393707 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.394141 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.394245 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:06:32.394550 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.382867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.1.127.252:46019)
0807 15:06:32.383067 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.383077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.383115 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:06:32.383700 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.383711 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.383780 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.383942 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.384937 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.386072 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.386976 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.387530 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.387694 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.390844 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.390853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.390854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.390863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.392809 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.393293 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.393296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.393296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.393372 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.393605 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.393605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.393606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.393889 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.394049 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.394154 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.394265 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260807 15:06:32.394827 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.384500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.254:37827 (local address 127.0.0.1:34168)
0807 15:06:32.384709 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.384746 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.384837 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.385035 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.385038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.385387 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.385519 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.385525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.386347 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.386348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.387289 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.387291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.387691 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.387692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.387753 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.388495 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.388506 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.390470 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.392558 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.392566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.392568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.392947 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.393222 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.393224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.393224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.393370 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.393722 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.393729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.393884 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.394106 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.394371 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260807 15:06:32.395506 25341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.254@KRBTEST.COM
I20260807 15:06:32.403213 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.395787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.253:32837 (local address 127.0.0.1:37058)
0807 15:06:32.395826 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.395835 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.395871 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.396151 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.396152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.396269 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.396320 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.396321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.397061 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.397062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.397594 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.397596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.398022 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.398023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.398094 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.398720 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.398725 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.399978 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.402076 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.402077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.402077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.402252 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.402502 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.402503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.402503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.402570 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.402919 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.402920 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.402972 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.403042 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.403143 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 15:06:32.403425 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.395869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.1.127.253:32837)
0807 15:06:32.395949 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.395951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.395962 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:32.395990 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.395991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.396071 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.396143 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.396401 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.396973 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.397680 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.397947 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.398036 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.400082 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.400084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.400084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.400094 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.401959 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.402332 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.402333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.402334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.402406 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.402643 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.402644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.402644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.402845 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.402918 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.403100 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.403190 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:06:32.403811 25341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:06:32.404155 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.395610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.1.127.254:37827)
0807 15:06:32.395652 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.395654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.395696 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:06:32.395772 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.395772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.395784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.395837 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.396527 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.396986 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.397770 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.397954 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.398035 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.400613 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.400614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.400615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.400622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.402276 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.402749 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.402750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.402750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.402809 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.403066 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.403067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.403067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.403216 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.403414 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.403873 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.403967 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.252@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.127.253@KRBTEST.COM
I20260807 15:06:32.412213 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.404158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46108)
0807 15:06:32.404209 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.404219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.404397 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.404630 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.404631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.404810 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.404852 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.404854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.405682 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.405683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.406305 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.406308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.406797 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.406798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.406854 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.407349 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.407354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.409033 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.411124 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.411125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.411125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.411345 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.411597 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.411598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.411598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.411656 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.411969 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.411970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.412008 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.412063 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.412142 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:06:32.412367 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.404265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.1.127.252:46019)
0807 15:06:32.404304 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.404306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.404316 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:32.404477 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.404477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.404488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.404539 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.404939 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.405590 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.406396 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.406656 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.406803 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.409108 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.409110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.409111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.409120 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.411033 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.411440 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.411442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.411442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.411520 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.411726 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.411727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.411728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.411890 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.411971 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.412148 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.412239 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:32.413725 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.404925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.1.127.253:32837)
0807 15:06:32.404971 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.404973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.404981 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:32.405017 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.405018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.405027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.405091 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.405545 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.406094 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.406934 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.407222 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.407321 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.410397 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.410399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.410400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.410409 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.412441 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.412854 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.412855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.412855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.412942 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.413173 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.413174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.413175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.413320 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.413502 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.413541 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.413622 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:32.414366 25341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: Bootstrap starting.
I20260807 15:06:32.415433 25341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:32.415781 1535 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:32.415804 25341 log.cc:824] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:32.416579 25341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: No bootstrap required, opened a new log
I20260807 15:06:32.416965 25161 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0b81546d18e240d2a4320cc7166208e4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34184
I20260807 15:06:32.418124 25341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.418252 25341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bc8df61d57411bbad0a991b8d76b12, State: Initialized, Role: FOLLOWER
I20260807 15:06:32.418340 25341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.418752 25341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:32.418740 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.418887 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:06:32.420919 25359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:32.420993 25359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:32.421195 25273 master_runner.cc:186] Error getting master registration for 127.1.127.254:37827: OK, instance_id { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" instance_seqno: 1786115192025401 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:06:32.423821 25273 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:06:32.430665 25266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 3
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.253@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:06:32.438562 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.431120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46114)
0807 15:06:32.431187 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.431200 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.431252 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.431396 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.431397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.431578 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.431625 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.431627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.432284 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.432285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.432811 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.432813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.433200 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.433201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.433261 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.433761 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.433767 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.435333 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.437260 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.437261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.437262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.437516 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.437738 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.437739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.437739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.437812 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.438179 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.438180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.438246 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.438337 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.438458 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:06:32.438561 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.431175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46114 (local address 127.1.127.252:46019)
0807 15:06:32.431219 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.431221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.431236 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:06:32.431319 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.431320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.431329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.431389 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.431710 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.432192 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.432908 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.433121 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.433203 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.435458 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.435460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.435460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.435467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.437163 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.437600 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.437601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.437602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.437665 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.437881 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.437882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.437882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.438095 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.438184 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.438378 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.438466 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:32.441277 25266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: Bootstrap starting.
I20260807 15:06:32.442128 25266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:32.442472 25266 log.cc:824] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:32.443512 25266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: No bootstrap required, opened a new log
I20260807 15:06:32.445654 25266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.445822 25266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1641b69c445047a4878277c947a53b31, State: Initialized, Role: FOLLOWER
I20260807 15:06:32.445916 25266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.446292 25266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:32.446246 25362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.446364 25362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:06:32.448381 25373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:32.448450 25373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:32.448662 25194 master_runner.cc:186] Error getting master registration for 127.1.127.254:37827: OK, instance_id { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" instance_seqno: 1786115192025401 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:06:32.451372 25194 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:06:32.474754 25187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } attempt: 3
Aug 07 15:06:32 dist-test-slave-8tdl krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115191, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.127.254@KRBTEST.COM for kudu/127.1.127.252@KRBTEST.COM
I20260807 15:06:32.483839 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.475334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.1.127.252:46019)
0807 15:06:32.475387 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.475389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.475413 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:06:32.475445 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.475446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.475459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.475526 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.476032 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.476918 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.478103 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.478324 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.478415 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.480767 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.480769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.480770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.480778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.482372 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.482858 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.482859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.482859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.482929 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.483187 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.483188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.483189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.483379 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.483453 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.483634 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.483722 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:06:32.483839 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.475149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.127.252:46019 (local address 127.0.0.1:46128)
0807 15:06:32.475210 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:32.475224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:32.475278 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:32.475594 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:32.475595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:32.475817 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:32.475867 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.475869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.477060 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.477062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.478005 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:32.478007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.478413 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.478414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:32.478477 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.479044 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:32.479050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:32.480668 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:32.482505 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.482506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.482507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.482789 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.483029 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:32.483030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:32.483031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:32.483102 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:32.483444 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:32.483445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:32.483508 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 15:06:32.483605 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:06:32.483721 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 15:06:32.486203 25187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: Bootstrap starting.
I20260807 15:06:32.487108 25187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:32.487473 25187 log.cc:824] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:32.488488 25187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: No bootstrap required, opened a new log
I20260807 15:06:32.490231 25187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.490419 25187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b81546d18e240d2a4320cc7166208e4, State: Initialized, Role: FOLLOWER
I20260807 15:06:32.490535 25187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.490973 25187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:32.491075 25161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:06:32.491133 25161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:32.492547 25161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.492854 25161 leader_election.cc:288] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [CANDIDATE]: Term 1 election: Requested vote from peers 1641b69c445047a4878277c947a53b31 (127.1.127.253:32837), 04bc8df61d57411bbad0a991b8d76b12 (127.1.127.252:46019)
I20260807 15:06:32.493443 25238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b81546d18e240d2a4320cc7166208e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1641b69c445047a4878277c947a53b31"
W20260807 15:06:32.493597 25388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:32.493650 25388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:32.493642 25238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:32.494902 25315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b81546d18e240d2a4320cc7166208e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04bc8df61d57411bbad0a991b8d76b12"
I20260807 15:06:32.495090 25315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:32.496294 25315 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b81546d18e240d2a4320cc7166208e4 in term 1.
I20260807 15:06:32.498394 25135 leader_election.cc:302] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bc8df61d57411bbad0a991b8d76b12, 0b81546d18e240d2a4320cc7166208e4; no voters:
I20260807 15:06:32.499902 25238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b81546d18e240d2a4320cc7166208e4 in term 1.
I20260807 15:06:32.502425 25118 master_runner.cc:263] No leader master found from master 0b81546d18e240d2a4320cc7166208e4
I20260807 15:06:32.502489 25118 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:06:32.490949 25376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.502808 25376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:32.502933 25376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:32.503049 25376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [term 1 LEADER]: Becoming Leader. State: Replica: 0b81546d18e240d2a4320cc7166208e4, State: Running, Role: LEADER
I20260807 15:06:32.503203 25376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } }
I20260807 15:06:32.503839 25376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b81546d18e240d2a4320cc7166208e4. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.503991 25376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:32.505702 25398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:32.506546 25398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 15:06:32.523087 25140 tablet.cc:2369] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:32.525377 25315 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [term 1 FOLLOWER]: Refusing update from remote peer 0b81546d18e240d2a4320cc7166208e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:32.525911 25394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:32.527844 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b81546d18e240d2a4320cc7166208e4. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.527956 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:32.529042 25238 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [term 1 FOLLOWER]: Refusing update from remote peer 0b81546d18e240d2a4320cc7166208e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:32.529412 25394 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:32.530913 25362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b81546d18e240d2a4320cc7166208e4. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.531031 25362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:32.536190 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.504974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.1.127.253:32837)
0807 15:06:32.505110 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.505112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.510897 (+ 5785us) server_negotiation.cc:407] Connection header received
0807 15:06:32.511036 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.511038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.511052 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.511088 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.515420 (+ 4332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.516296 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.522923 (+ 6627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.523271 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.523331 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.532343 (+ 9012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.532345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.532345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.532354 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.534650 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.535208 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.535209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.535210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.535277 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.535495 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.535496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.535496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.535654 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.535747 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.535972 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.536080 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 15:06:32.538636 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.504444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.1.127.254:37827)
0807 15:06:32.504501 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.504503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.507584 (+ 3081us) server_negotiation.cc:407] Connection header received
0807 15:06:32.507706 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.507707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.507719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.507753 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.508697 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.509538 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.511960 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.512310 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.512361 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.519739 (+ 7378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.519741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.519741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.519752 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.521867 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.524022 (+ 2155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.524024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.524024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.524106 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.524620 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.524622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.524622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.524781 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.524876 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.538402 (+ 13526us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.538515 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:06:32.544552 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:32.505417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.1.127.252:46019)
0807 15:06:32.505461 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.505463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.513059 (+ 7596us) server_negotiation.cc:407] Connection header received
0807 15:06:32.514567 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.514568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.514584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.514620 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.517526 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.518345 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.537319 (+ 18974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:32.537697 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:32.537747 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:32.541086 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:32.541088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:32.541089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:32.541099 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:32.543043 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.543527 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.543528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.543529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.543594 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:32.543857 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:32.543858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:32.543859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:32.544031 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:32.544124 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:32.544349 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 15:06:32.544440 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:32.547880 25394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.548040 25394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:32.548847 25362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.549090 25362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:32.551225 25402 mvcc.cc:205] Tried to move back new op lower bound from 7315927828559851520 to 7315927828494622720. Current Snapshot: MvccSnapshot[applied={T|T < 7315927828559851520}]
I20260807 15:06:32.551517 25405 mvcc.cc:205] Tried to move back new op lower bound from 7315927828559851520 to 7315927828494622720. Current Snapshot: MvccSnapshot[applied={T|T < 7315927828559851520}]
I20260807 15:06:32.552733 25376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.552850 25376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:32.553347 25347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b81546d18e240d2a4320cc7166208e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b81546d18e240d2a4320cc7166208e4" member_type: VOTER last_known_addr { host: "127.1.127.254" port: 37827 } } peers { permanent_uuid: "1641b69c445047a4878277c947a53b31" member_type: VOTER last_known_addr { host: "127.1.127.253" port: 32837 } } peers { permanent_uuid: "04bc8df61d57411bbad0a991b8d76b12" member_type: VOTER last_known_addr { host: "127.1.127.252" port: 46019 } } }
I20260807 15:06:32.553462 25347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:06:32.554126 25403 mvcc.cc:205] Tried to move back new op lower bound from 7315927828559851520 to 7315927828494622720. Current Snapshot: MvccSnapshot[applied={T|T < 7315927828559851520}]
I20260807 15:06:32.556463 25398 catalog_manager.cc:1347] Generated new cluster ID: 7eea933985de450db7dd9d8d1529e56c
I20260807 15:06:32.556536 25398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:32.587178 25398 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:32.588169 25398 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:32.605453 25398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b81546d18e240d2a4320cc7166208e4: Generated new TSK 0
I20260807 15:06:32.605808 25398 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:06:32.638294 25190 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.636946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.1.127.254:37827)
0807 15:06:32.636995 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.636997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.637506 (+ 509us) server_negotiation.cc:407] Connection header received
0807 15:06:32.637598 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.637600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.637610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.637640 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.638210 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34198: BlockingRecv error: recv got EOF from 127.0.0.1:34198 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:06:32.638298 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.637383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.1.127.253:32837)
0807 15:06:32.637445 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.637447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.637459 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:32.637491 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.637492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.637505 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.637574 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.638227 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37092: BlockingRecv error: recv got EOF from 127.0.0.1:37092 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 15:06:32.639202 25344 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.637385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.1.127.252:46019)
0807 15:06:32.637636 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.637639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.638551 (+ 912us) server_negotiation.cc:407] Connection header received
0807 15:06:32.638656 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.638658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.638670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.638723 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.639105 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46144: BlockingRecv error: recv got EOF from 127.0.0.1:46144 (error 108)
Metrics: {"server-negotiator.queue_time_us":223}
W20260807 15:06:32.640811 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.640137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.1.127.253:32837)
0807 15:06:32.640176 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.640178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.640272 (+ 94us) server_negotiation.cc:407] Connection header received
0807 15:06:32.640327 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.640328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.640348 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.640421 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.640759 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37102: BlockingRecv error: recv got EOF from 127.0.0.1:37102 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 15:06:32.643597 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.642995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.1.127.253:32837)
0807 15:06:32.643045 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.643046 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.643057 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:32.643124 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.643125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.643136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.643208 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.643545 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37114: BlockingRecv error: recv got EOF from 127.0.0.1:37114 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:06:32.647661 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.646892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.1.127.253:32837)
0807 15:06:32.646929 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.646931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.647137 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:06:32.647167 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.647168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.647179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.647226 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.647601 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37124: BlockingRecv error: recv got EOF from 127.0.0.1:37124 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 15:06:32.653643 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.652853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.1.127.253:32837)
0807 15:06:32.652906 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.652908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.653029 (+ 121us) server_negotiation.cc:407] Connection header received
0807 15:06:32.653102 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.653102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.653113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.653210 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.653580 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37130: BlockingRecv error: recv got EOF from 127.0.0.1:37130 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:06:32.663931 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.663122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.1.127.253:32837)
0807 15:06:32.663191 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.663194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.663293 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:06:32.663366 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.663367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.663382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.663444 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.663850 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37142: BlockingRecv error: recv got EOF from 127.0.0.1:37142 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:06:32.681984 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.681231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.1.127.253:32837)
0807 15:06:32.681313 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.681315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.681383 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:06:32.681438 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.681439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.681453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.681540 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.681909 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37154: BlockingRecv error: recv got EOF from 127.0.0.1:37154 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 15:06:32.716512 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.715595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.1.127.253:32837)
0807 15:06:32.715691 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.715693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.715804 (+ 111us) server_negotiation.cc:407] Connection header received
0807 15:06:32.715899 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.715900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.715914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.715997 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.716427 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37170: BlockingRecv error: recv got EOF from 127.0.0.1:37170 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 15:06:32.783061 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.782105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.1.127.253:32837)
0807 15:06:32.782208 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.782210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.782333 (+ 123us) server_negotiation.cc:407] Connection header received
0807 15:06:32.782370 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.782371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.782383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.782456 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.782975 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37174: BlockingRecv error: recv got EOF from 127.0.0.1:37174 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 15:06:32.913353 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:32.912458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.1.127.253:32837)
0807 15:06:32.912577 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:32.912579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:32.912707 (+ 128us) server_negotiation.cc:407] Connection header received
0807 15:06:32.912811 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:32.912812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:32.912827 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:32.912906 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:32.913283 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37188: BlockingRecv error: recv got EOF from 127.0.0.1:37188 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 15:06:33.171820 25270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:33.170858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.1.127.253:32837)
0807 15:06:33.170962 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.170965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.171128 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:06:33.171248 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.171249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.171262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.171343 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:33.171749 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37196: BlockingRecv error: recv got EOF from 127.0.0.1:37196 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:06:33.421494 25359 catalog_manager.cc:1259] Loaded cluster ID: 7eea933985de450db7dd9d8d1529e56c
I20260807 15:06:33.421582 25359 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: loading cluster ID for follower catalog manager: success
I20260807 15:06:33.422461 25359 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: acquiring CA information for follower catalog manager: success
I20260807 15:06:33.422817 25359 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04bc8df61d57411bbad0a991b8d76b12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:06:33.448954 25373 catalog_manager.cc:1259] Loaded cluster ID: 7eea933985de450db7dd9d8d1529e56c
I20260807 15:06:33.449072 25373 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: loading cluster ID for follower catalog manager: success
I20260807 15:06:33.449924 25373 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: acquiring CA information for follower catalog manager: success
I20260807 15:06:33.450251 25373 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1641b69c445047a4878277c947a53b31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:06:33.688282 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:33.685259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.1.127.253:32837)
0807 15:06:33.685440 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.685442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.685501 (+ 59us) server_negotiation.cc:407] Connection header received
0807 15:06:33.685608 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.685609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.685690 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.685783 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:33.685946 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.686525 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.687415 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.687625 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.687710 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:33.687894 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:33.688018 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:33.688110 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:33.688136 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:06:33.688172 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260807 15:06:33.693293 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:33.690507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.1.127.252:46019)
0807 15:06:33.690668 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.690671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.690716 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:06:33.690765 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.690766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.690822 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.690895 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:33.691033 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.691657 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.692509 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.692747 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.692804 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:33.692944 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:33.693056 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:33.693110 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:33.693152 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:06:33.693170 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260807 15:06:33.698267 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:33.695384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.1.127.254:37827)
0807 15:06:33.695532 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.695535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.695549 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:06:33.695603 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.695604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.695665 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.695772 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:33.695913 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.696540 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.697337 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.697550 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.697629 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:33.697852 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:33.697985 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:33.698047 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:33.698134 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:06:33.698167 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260807 15:06:33.698774 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:33.695645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.1.127.252:46019)
0807 15:06:33.695703 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.695705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.695785 (+ 80us) server_negotiation.cc:407] Connection header received
0807 15:06:33.695966 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.695967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.695979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.696040 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:33.696179 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.696898 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.697930 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.698229 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.698294 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:33.698484 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:33.698518 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:33.698543 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:33.698648 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:06:33.698701 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:06:33.699151 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:33.695383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.1.127.253:32837)
0807 15:06:33.695500 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:33.695502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:33.695671 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:06:33.695818 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:33.695819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:33.695831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:33.695863 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:33.696044 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.696819 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.698065 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:33.698345 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:33.698422 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:33.698642 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:33.698706 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:33.698754 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:33.699024 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 15:06:33.699060 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 15:06:33.700572 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 25118
I20260807 15:06:33.708657 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 25194
I20260807 15:06:33.715044 1535 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbmaIkX/build/release/bin/kudu with pid 25273
2026-08-07T15:06:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3145 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6143 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291151 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST